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FOREWORD 


For several reasons, the publication of this report was postponed for several years. 
Its contents were summarized at the time of this author’s oral presentation at DTRC in 
July 1985, reflecting the state-of-the-art up to 1984. The manuscript of the written 
version was completed in August 1987 and is now to be published five years later in 
1992. During these years the state-of—the—art has made considerable progress, and this 
author finds the “review” to be insufficient, especially in Part III because of the recent 
works of several researchers. In order to update the report this author has added a 
“Supplement of References,” (Refs. 101-124) listing publications that have appeared 
since 1984, together with some other publications that he had failed to refer to in the 
original manuscript. 


THIS PAGE INTENTIONALLY LEFT BLANK 


PREFACE 


The David W. Taylor Lectures were initiated as a living memorial to our founder in 
recognition of his many contributions to naval architecture and naval hydrodynamics. 
Admiral Taylor was a pioneer in the use of hydrodynamic theory and mathematics for the 
solution of naval problems. The system of mathematical lines developed by Taylor was 
used to develop many ship designs for the Navy long before the computer was invented. 
He founded and directed the Experimental Model Basin; perhaps most important of all, 
he established a tradition of applied scientific research at the “Model Basin” which has 
been carefully nurtured through the decades and which we treasure and protect today. In 
the spirit of this tradition, we invite an eminent scientist in a field closely related to the 
Center’s work to spend a few weeks with us, to consult with and advise our working staff, 
and to give a series of lectures on subjects of current interest. 


Our tenth lecturer in this series is Dr. Yasufumi Yamanouchi who is currently 
Technical Advisor to Mitsui Engineering and Shipbuilding Co., Ltd. Dr. Yamanouchi 
graduated from the Tokyo University in 1943 and received his Dr. Eng. from there in 
1962. For many years (1946—73), he served with the Governmental Research Institute on 


Ship Technology, that changed its name from Railway Technical Research Institute 
(1946-50), then to Transportation Technical Research Institute (1950-63), and finally to 
Ship Research Institute, Ministry of Transport. He was Head of the Ship Performance 


Division (1962-63) and Head of the Ship Dynamics Division (1963-69), then Deputy 
General Director (1969-72) and finally Director-General of the Ship Research Institute 
(1972-73), Ministry of Transport in Tokyo, Japan. He joined Mitsui Engineering and 
Shipbuilding Co., Ltd. as Senior Deputy Director, advancing to Technical Director upon 
his retirement in 1983. He contributed to the construction and organization of a new 
research laboratory, and was the first General Manager of the Akishima Laboratory of 
Mitsui from 1975 to his retirement in 1983. He was Professor at the College of Science 
and Technology (1982-89) of Nihon University in Tokyo, Japan. 


Dr. Yamanouchi has had a long and distinguished scientific career, with pioneering 
publications on ship dynamics, ocean waves and stochastic processes. He is no stranger 
to the United States, having done research at the Davidson Laboratory of Stevens Insti- 
tute of Technology from 1957-60 and having served as Visiting Professor at the U.S. 
Naval Academy during 1984-85. We are most honored that he agreed to be a David W. 
Taylor Lecturer. 


ABSTRACT 


In Part I, the general procedures of the conventional method of correlation or 
spectrum analysis of a random process (nonparametric method) are reviewed, stressing 
the statistical reliability of the results. A few suggestions for improving the coherencies 


The model-fitting technique supported by AIC criteria is introduced, with the examples 
of application to the seakeeping data. In Part II, the statistical treatments of nonlinearities 
in random process analysis are summarized and reviewed. Conclusions are given, and 
future work is proposed. 


CHAPTER 1 
INTRODUCTION 


1.1 PROLOGUE 


In the summer of 1951, a group of research naval architects who were scheduled to 
be on board the cargo ship Nissei—Maru cruising the Pacific from Japan to the United 
States was discussing the design of a system for measuring, recording, and analyzing ac- 
tual ship performance data. Such a system was needed for the first large-scale 
cooperative test,! organized by the Japan Towing Tank Committee (JTTC) of the Society 
of Naval Architects of Japan (SNAJ) and scheduled to start at the end of the year, to pro- 
vide a basis for the post-World War II study of hydrodynamic ship performance in all the 
laboratories and universities in Japan. 


To find the relationships between ship’s propeller revolutions, shaft horsepower, 
modes of motion, rudder angle, and so on, we decided to record, simultaneously and as 
accurately as possible, as many elements of ship performance as possible. Such measure- 
ments would provide an overview of the response of the ship. Many practical limitations, 
such as budget constraints, had to be considered, however. In addition we didn’t know the 
real limits of the accuracy of these kinds of measurements, nor did we know how to 
choose the proper duration of observation times. After lengthy discussion, it was decided 
to record each type of measurement for three minutes. 


Very few naval architects at that time had a good knowledge of probability or of 
statistics. We did not know how to analyze the data taken at sea, and we did not know 
how to estimate the ensemble characteristics of performance from a single sample obser- 
vation. The simultaneous records of the averages of the 3—minute responses gave us 
valuable knowledge of actual sea performance, but we realized that simple averages 
sometimes cover or conceal important information. For ship oscillations, of course, mean 
values are not significant, but this author could not be satisfied with merely noting the 
average frequency and mean amplitudes. 


During the test, the author? was responsible for measuring the ship’s relative speed 
and developed a new type of ship speed meter, based on an idea of Dr. Shiha. This new 
meter measured the frequency of Karman vortices produced behind a triangular cylinder 
towed by the ship and gave instant indications of speed variations. Although the towing 
point was selected to minimize the effects of pitching, heaving, and rolling, these motions 
did affect the speed of the towed body. The objective was to eliminate the effects of all 
motions except surging and to derive from these records, by some method, the real varia- 
tions in the ship’s speed relative to the water. The author later found that an advanced 
technique of multiple input analysis would have been useful. 


After the test at sea, we continued to look for methods for analyzing the data and 
found a stochastic process analysis used in weather forecasting to predict temperature, 
humidity, precipitation, and so on. The author started at once to study time series analysis 
and tried first several kinds of periodogram analysis and then correlogram and spectrum 
analysis.? Because of the poor communications in the engineering and scientific fields in 
Japan at that time (even five years after the war) we had very little information about the 
outside world until 1954 and were unaware of the pioneering work of Dr. M. St. Denis 


and Prof. W. J. Pierson Jr.4 in 1953. We had to start on our Own independent of the 
achievements in the USA. 


1.2 PURSUING THE IMPROVEMENT OF COHERENCY FUNCTIONS 


After the bases for the mathematical and statistical theories were established by N. 
Wiener,>-° J. L. Doob,’ C. E. Shannon, §, O. Rice,’ and others, mostly in the 1940’s, the 
stochastic analysis techniques were applied in many scientific and engineering fields 
besides communications and control engineering. These techniques were adopted rather 
early in the analysis of ocean waves and of a ship’s response at sea. This method has been 
pretty well formulated by the efforts of oceanographers like G. Neumann,!° W. J. Pierson, 
Jr.,! M. S. Longuet-Higgins,!2 D. E. Cartwright and M. S. Longuet—Higgins,!3 and by 
the pioneering work of M. St. Denis and W. J. Pierson, Jr.,4 succeeded by E. V. Lewis,!4 
and J. F. Dalzell and Y. Yamanouchi,!5 and is now rather popular with us. We now know 
that, in practical applications and when applying certain theories, a few statistical consid- 
erations are necessary in the numerical computations in order to get reliable results. 


In Part I, the author tried to show the problems encountered in sample computa- 
tions, in the so—called correlation method that are also closely related to the basis for 
model fitting techniques (that is, the parametric method treated in Part II) and reviews the 
conventional nonparametric method. The author did not intend to go into detail about the 
analysis technique. 


The coherency function, if properly calculated, is a good index of the extent to 
which spectrum analysis, as a linear process, is valid for application to a stochastic pro- 
cess. A few results of this author’s efforts in this field, presented later in Part I, are related 
to improvement of the techniques for obtaining a good estimate of coherencies. 


1.3 TIME DOMAIN CHARACTERISTICS 


Time series analysis is sometimes called spectrum analysis, thus showing that esti- 
mation of reliable spectral functions is very important in the analysis of time series. 


Spectral functions are surely powerful functions which provide good information on 
a stochastic process and also resolve the tangled relations of convoluted types in the time 
domain. However, because of this fact, and also partly because (auto) correlation did not 
appear in an important way, in St. Denis and Pierson’s pioneering paper,’ rather little at- 
tention has been paid by naval architects to the time domain relations over the past few 
decades. From his first involvement in this study, this author has thought that time do- 
main functions deserved more attention and has made some efforts along this line. 


Of course spectral functions and correlation functions are actually the same function 
expressed in different ways. Sometimes, however, in sample computations, applying the 
time domain expression helps us understand the characteristics of the stochastic process 
better, because we are accustomed to expressing the physical process by differential equa- 
tions that are expressions in the time domain. The time domain expression of stochastic 
processes helps us in the analysis to use already known characteristics of the process. 
Moreover, the correlation window Operation in the sample computation is just a multiply- 
ing operation, whereas the spectral window operation is now an entangled convolution 
operation that is more complex in real sample computations. In sample computations, 


consideration of the windows is very important and also a troublesome problem to get 
reliable results. 


Parametric estimation of the spectral function, explained in Part I, is a method of 
fitting a certain statistical model to the process to be analyzed and then estimating the 
parameters of that model. This method is based on time domain characteristics, and the 
model fitted is closely related to the equations of motion of the process itself. The method 
represents a different approach to spectrum analysis and, this author believes, that is a 
promising one, supplementing the conventional nonparametric approach, treated in Part I. 
So, in Part II, several types of discrete parameter models will be introduced in detail, and 
then the results of application of the model fitting method to the analysis of seakeeping 
data will be shown. By this method, the actual response system, in which the output is fed 
back to the input to some extent, can also be tackled. This kind of system has been hard to 
be analyzed by the conventional method. 


1.4 TREATMENT OF NONLINEARITIES 


One of the reasons for low coherencies in linear spectral analysis is the existence of 
nonlinearities in the response or in the input. So in Part II, the treatment of nonlinearities 
in process analysis is reviewed. One of the greatest achievements in this field for the anal- 
ysis of seakeeping data is due to John F. Dalzell!®1” in the application of polyspectra. 
This application will not be described in detail, except the basic idea, of this treatment but 
another review of the treatment of nonlinearities will be given, to make clear the mutual 
relationships of these several different approaches to the problem. In addition, the treat- 
ment of nonlinearity in response characteristics during one trial has been introduced in 
Section 3.4 in Part I, as an example of multi—input single-output analysis. 


1.5 SCOPE OF STUDIES 


In statistical studies on seakeeping, there are roughly two kinds of applications. One 
is based on the invariant characteristics of a ship itself, and its behavior is studied statisti- 
cally, assuming the excitement from the environment is also stationary. The other 
involves the macroscopic probabilistic distributions of seakeeping behavior, assuming a 
variety of changes in environmental conditions. The former is sometimes called short— 
term statistics, and the latter, long—term statistics. They are, of course, closely related, and 
the short-term statistics are usually used as the basis for studying the long-term statistics. 
Here mostly short—term statistics will be treated. 


PARTI 


A REVIEW OF SPECTRAL ANALYSIS THROUGH PERIODOGRAM 
(NONPARAMETRIC SPECTRAL ANALYSIS) 


To clarify the problems and difficulties encountered in sample computations and 
make them the basis for further discussions, the rough scheme of the techniques of spec- 
trum analysis (through periodograms, the popular nonparametric method) will be 
reviewed first in Chapter 2. Then some ideas proposed by this author for solving these 
difficulties will be summarized in Chapter 3, Part I. Many text books,!*3 especially the 
comprehensive one by Priestley,”3 were used as references in Chapter 2. 


CHAPTER 2 


BASIC PROCEDURE OF THE SPECTRAL ANALYSIS AND 
THE PROBLEM OF SAMPLE COMPUTATIONS 


2.1 RANDOM PROCESS AND ITS CHARACTERISTICS 


Here the general continuous process on f is expressed by X(f), its realization as x(t), 
and the probability density distribution function related to this process as p,(x). The gen- 
eral discrete process is X,, its realization is X;, and probability distribution density function 
is p;(x). Then, as expected values, 


foe) 


mean [X(1)] = E [X(.)] = | x(t)p(x)dx = M(t) (2.1) 


—o 


var. [X()] = E [oo zi ac} | = | L()-BwO!P pixdx = 07(t). (2.2) 


—o 


Usually, the probability density distribution function P(x) is a function of t ; accord- 
ingly, the mean u(r) and the variance o°(t) are also functions of time. 


Joint probability density distribution functions, Prytgty .. .4,(41,%2,X3, . . . Xp) Exist 
for all n,n =1,2,...n,(n—> oo) and all p,,(x1),p,(x2), .. . > Pyn%1%2), ... 65 
Prytyt,(%1,%X2,X3), ...5 .... are defined as their marginal functions. 


Theoretically, if all the joint distribution functions of all orders 


P1,(%1), Pr(X2), oo 4 
JD KC SO Ma CeO) 2 oo < 


[DeeCSio2eH3S))y 6 0 0 0 


[Din 0 3 o EACSIn heey oo o Beh 00 0900 
are known, the probability structure of X; is completely specified. 


2.1.1 Completely Stationary 


If 
Put 00-0 fr (X1,X2, 0. 950 XP) = Pustkh+k, 09-0 t,+k, ’ Gare: 9.0.0 0 a). (2.3) 
for any f},t2, .... t, and any k, this process is completely stationary. 


2.1.2 Stationary Up to Order m 


In this case, the joint moment up to order m should be the same. 


E[{xcen)]™(XC)}™ [x¢en)} 
= E[Ke1 + HIXea +H"... xe, +0)", (2.4) 
for all positive integers m, m2, ..™, 
where 
mt+m+....m,S™mM. 


2.1.3 Stationary Up to Order 2 


Especially when the order is m = 2, the process is called weakly stationary. When 
the probability density distribution functions are Gaussian, they are completely deter- 
mined by the means, variances, and covariances of two variables, and accordingly they 
are completely stationary. 


2.1.4 Ergodicity 


When the ensemble of the averages across the processes converge to the corre- 
sponding time averages along the process over period N (when N tends toward infinity 
and the mean square is consistent), the process is called ergodic. The ergodicity is a more 
strict condition than the stationality as is shown in Fig. 2.1. 


EVOLUTIONARY 


STATIONARY UP TO 1ST ORDER 
ERGODIC UP TO 2ND ORDER 


STATIONARY UP TO 2ND ORDER 


COMPLETELY STATIONARY 
AND ERGODIC 


Fig. 2.1. Stationarity and ergodicity. 


For example, 


R(t) =E [X(t) -X(t+7)] 


| | X(t)X(t + T) Pt, X147)AX1AX 141 


T 
1 
lim) | x(t)x(t + T)dt. 
Be GP Ir (2.5) 
2.1.5 Summary of Gaussian (Normal) Probability Distribution Functions 


For convenience, the Gaussian probability distribution functions for various num- 
bers of variables are summarized here. For a single variable, 


1 1 @aonJF 
p() = (Qno2V2 ex - 5 aie j (2.6) 
Normalized by z = (x—fx)/0,, 
(Ne ee eS 2.7) 
zZ) =—— €?. 3 
Nay eae 


For two variables X and Y 


6s yo Ea ce oles mesial MON 
PY 2n|0203(1 - 9”)|"” 2 (1-07)0? (1-97)o,0y (1-07)02 
seal 1 | oy > 200,07 Ce ie 
= Se exp | - ry |S (x-,) - A (x — Lx) — My) + i (y—Ly) 


1 
> Ona? “*P |- 5 Bu —H.)? — 2Barx— M2) Mr) + Art} (2.8) 


Here, 
A Yxx Yxy |] _ |] oF  0020y |] = 0205-07 0305 (2.9) 
Yyx Yoy 000, a5 = 0703(1 -@°), 
ee SIVKOM See, Me, BS BRK) = wou, 
Yy = EIY-Y] = 05 , yx = EY -X] = Qoyo,. (2.10) 
| | = Bax Pry = abi = a Yyy ~ Vy = ite a5 ycoe2y 
2 WB el” tee Plays | yes) Meso, (lool lool nasal 
(2.11) 
For n variables X,,X2, ....... Xn, 
DOGS XD A eens =e en) a eklias ne! exp I ie SNe) (2.12) 
pit (2ny"/2A1/2 Dae a | 


where 


Y= {o; (2.13) 


and a'/ is an element of inverse matrix, so that 


1 
JDERedoy 5S 6 6.0 Xn) exp -5{e-#) D1 @-m)| . (2.14) 


1 
= (27)"/2A1/2 


2.2 PROPERTIES REQUIRED FOR ESTIMATOR 
The following properties are required for the estimator of some statistical value. 


2.2.1 Unbiasedness 


As the number of samples n tends toward infinity, for the estimator 6 of real 6, 


bias) = {E() es 6| 5 (0), 3 (2.15) 


22.2 High Relative Efficiency 
If we suppose that both estimators 6,,6, are unbiased, then the higher the 
rel. effic. = var. (6;)/var. (62) (2.16) 


is, the better estimator 6, is than @,. 


2.2.3 Small Mean Square Error 
Mean square error is defined as 


M*6) = E[(@ - 6)"] 
= A|{6 z E(6)| + {E(6) e 6} 
= A|{6 as z6)} | + {E(6) bs a" + 2{E(6) - o\zI6 ~ E(6)] 
= var. (6) + b%(6), (2.17) 


which should be small. When M2(6 ;) < M(6>), we adopt 6 as better than 6. 


2.2.4 Consistency 
As the number of the samples n tends toward infinity, var. (6) — 0, and bias 


b2(6) — 0 must be satisfied. Then the mean square error M2(6) — 0 also stands and is 
called the “mean square consistency.” 


225 Sufficiency 


The estimator @ must contain all the information X;,X2, . .. . X, in the sample, 
relevant to the estimator of 6 


(30 Cano, sacs SO 


2.3 AUTOCOVARIANCE FUNCTION AND ITS ESTIMATES 
When the process X(t) is stationary up to an order of 2, the covariance function 


cov. [X(t) X(t+7)] = E [|X(t)—pl{Xr+7)-u}] = R@ (2.18) 


is a function oft only. Then, 0(t) = R(t)/R(O) is called an autocovariance coefficient, 
and 


RO) = E [{X()—-pw}*] = var. [X()] = 07. (2.19) 


When the process is real valued, R(—T) = R(t) as in Fig. 2.2, and when the process is 
complex valued, R(—T) = R*(t). This function R(z) is a measure of similarity in a sense, 
and is also a measure of the memory of the process. R(t) plays a big role later, in the 
parametric analysis in Part II, in identifying the statistical model that will fit the process. 


Meany, varianceo*, R(t), and o(t) are constant by t. 


{ R(t) 


Fig. 2.2. Autocovariance function. 
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23.1 Estimates of R(r) 
For the discrete parameter process x; , 


R(r) = E [(X&:-4) X14 -—Mrr)] 
=F [X; Xr] 5 (2.20) 
when £; = Uy, = E[X;,] = 0. 


2.3.2 Unbiased Estimate 
Instead of averaging the ensemble, take the time average as 


N-Irl 
1 
FO) ae 2 (X,—X) Xr —X) (2.21) 
WHET 7a) () Se-t-teil et + (N-1). 


If we ignore the effect of estimating u by X,then R(r) is an unbiased estimate of 
R(r), because 


i 1 N-irl 
BLS) Ora > EX: -wXur-w) 
—Irl =i 
N-trl 
= SY Rr) = SRR) (2.22) 
N-Irl = N-tIrl 
2.3.3 Biased Estimate 
On the contrary, 
r N-—rl 
RO) = = XW) Xr — 1) (2.23) 
t=] 
is a biased estimate, because 
n | fe 1 Ir 
E[R =— = — (N-Irl) R(r) = —_— 
[R)] + Re "7 rl) Rr) (: re 
= R(r) ue R(r) 2.24 
a N ’ ( . ) 
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biased by (I7l|/N) R(r). WhenN — ©, E[R(r)] — R(r), and is therefore asymptotically 
unbiased. 
By statistical mathematics, we can get 


var. [R°(r)] > O(1/(N=Irl)) (2.25) 
var. [R(r)] > O(1/N). (2.26) 


1. When r is small relative to N, the difference between R%(r) and R(r) is 
small, and the bias of [R(r)] is also small. 
2. When r becomes large relative to N and approaches N-1, 


bias [R(n)] — R(r). However, whenr > ~, R(r) — 0. Therefore, when 
N is very large, the bias remains small at all r. 
3. When r > (N-1), 


var. [R(r)] — 0(1) (2.27) 


var. [R(r)] — O(1/N). (2.28) 


Therefore, the tail of correlation Rr) shows a wild and erratic behavior. Also, from 
Statistical mathematics, cov. [R(r) R(r + s)] was computed and fairly high correlations 
between neighboring points were found when s is small. When 7 tends toward infinity, 
R(r) — O. It can be concluded here that Rr) will be less damped than R(r) and will not 
decay as quickly as R(r). 


2.4 SPECTRUM ANALYSIS 
The spectrum function decomposes a time varying quantity into a sum (or integral) 
of sine and cosine functions. 
2.4.1 Spectrum for Various Processes 
a. For Deterministic Periodic Functions—Fourier Series 
For periodic functions with period T, 
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1 2 t _ (ant 
X(t) = B ao + » an cos( =) + b, sin( =) (2.29) 


where 
I 
1 | Xo) Jtnt a 
n= cos — 
ser T 
ie 
: (2.30) 
2 
t 
pone [ xo fin ae 
T T 
ae Ta 
2 
The a, 


and 6, functions are called Euler—Fourier coefficients. Equation 2.30 is 


t t 
based on the orthogonality of cos, sin. 


For the existence of a,, b,, and for the convergence of the series, the conditions 


I I 
2 2 
IX(t)ldt < «, | IX(t)P?dt < © (2.31) 


RIN 


are sufficient. 


1 


1 
Replacing a,b, by Co =o, Cp = E (a? + Ale gives 


the total energy [-T/2 to T/2] as 


I 


Sl 1 
| X(t = ST 7% +5 (a + bi =T c3 (2.32) 
n=1 


ae ea 
2 


and the total power [- 7/2 to T/2] as 


13 


eas (2.33) 


c? is the contribution of the nth term to the total power, as in Fig. 2.3. 


i POWER 


2 3 
VO T 
Fig. 2.3. Power spectrum of periodic function. 


b. For Nonperiodic Deterministic Functions—Fourier Integral 
For nonperiodic functions with finite energy as in Fig. 2.4, 


X(t) = | X(@) edu , (2.34) 
X() == | X(t) © dw . (2.35) 


Fig. 2.4. Nonperiodic function with finite energy. 
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For the existence and convergence of this expansion, conditions 


| IX(t)l dt < 0, | x(n}? dt < © (2.36) 


are necessary. In this case, Parseval’s Relation is 


| X°(t)dt = 21 | X(w)*dw. (2.37) 


Here, as in Fig. 2.5 271X(w)|7dw is the contribution to the total energy of those 
components in X(t) whose frequency lies between w and w + dw. Accordingly, 
27lX(w)I? denotes the energy density function. In this case determining the power makes 
no sense, because when T — ©, from Eq. 2.36, power tends to 0. 


| 2n|X(co)|* 


Fig. 2.5. Energy spectrum of nonperiodic function. 


c. For Stationary Stochastic Processes 
For a single realization x(t) of X(t), as in Fig. 2.6 assume E[X(1)] = 0; X(t) is sto- 
chastically continuous. [R(0) = ¢2,0(0) = 1] 


X (t) 


“SP e) T 


Fig. 2.6. Stationary stochastic process with finite power. 
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Since there is no periodicity, no Fourier series exists, and because of stational- 


ity, | LX(1)ldt < © does not hold, so X(t) does not possess a Fourier integral. Thus, 


2x lim LX(w)!? will not exist and may become infinity. So here 


X(t) -T<t<T 


Xt) = 2.38 
rt) 0 otherwise Gee) 
and the following process is introduced: 
X7(t) = | X7(w)e™'dw 
) T 
1 ; 1 ; 
xX =— | XndeMdt=— | X(Ne dt. 2.39 
TO) = | mt)e = | (the (2.39) 
—0 —T 


Then, since the power is 27IX7(w)I*/T instead of 27lX7(w)I?, which makes T > ©, 
_ 2nIXr(w)? . 

lim Ton is for the total process. The expected value of this power is 

2X 1(w)!* 

2nXro)I" (2.40) 


E [Power] = lim E 
Th 


and is defined as s(w). By manipulation, 


aiXHw yh LL 
Ge OE [2xX7(w)] [2xX7(w)] 
CE ne | X7(t)e"at | Xr(t—t)e"™ at 
Oe Ye 
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1 ] : 
— — | Xt) Xr(t—7t)dt |e" dt 
om | oT | mt) X7(t-T) 


1 
=— | Rt) dr, 
= | TT )e 


*Denotes the complex conjugate. 


Here, 


Rr(t) = im cn | XOX Mt —1)dt. 


Accordingly, for the total 


E [P = s(w) = limE 
[Power] (@) oF 


1 7 
= lim — | eE[Rd1)]dt. 
Hie Se | [Rr{T)] 


Since X7(t) was defined as X(t) for —T < t <T, and otherwise 0, 


Ti 
i on | X(t) X(t—Itl)dt, Il<T 
R(t) = 4 2T 
+(T-trl) 
0 i>T 
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271X(w)I? 


(2.41) 


(2.42) 


(2.43) 


(2.44) 


Th T 
Elk) == | E[X(t) X(t+)] dt= | R(x)dt 


(T-Irl) (T-Ir1) 
| 
R(t)} 1 Cah Itl<T 
= OE : (2.45) 
0 Intl > T 
As T approaches infinity, 
T 
1 itl 
= lim) <—— 1-— | e™’R(t)d 
s(@) Jin {= | ar | ° (t)dt 
—T 
= idx | eat R(t)dt. (2.46) 
20 


For the existence of s(w), X(t) must have an autocovariance function R(r) that is 
continuous everywhere, including at t = 0, and 


R(t) = | eT dS(w) (2.47) 
o(t) = | eT dF(w). (2.48) 


This relation is called Wiener—Khinchine’s Theorem. Here, F(q) is the normalized 


integrated spectrum S(w)/o2, where if s(w) is continuous and smooth, 


S(@) = | soya , and F@)= | fede, (2.49) 


where f(w) = s(w)/o2. 


18 


The properties of F(w) and f(w) correspond to those of the probability density and 
probability density distribution functions, P(w) and p(w), for continuous distribution. 


2.4.2 Spectral Representation of the Stationary Random Process 


Instead of using the Fourier integral, the stationary process is more generally ex- 
pressed by the Fourier Stieltjes integral as 


X(t) = | e! dZ(w), (2.50) 


where dZ(w) is in the order of O(Vdw) and is larger than dw. dZ(w) and dZw ) are un- 
correlated, that is they are orthogonal to each other 


5 [iaz(w y?] = dS). (2.51) 


When the process has a purely continuous spectrum, from Eq. 2.49, rewritten as 


dS(w) = s@@) da, (2.49°) 
E [iaz(wy?] = s(w)dw = AS(w). (2.52) 

For a real valued process, 
R(-T) = R(t), (2.53) 


and 


1 1 
s(w@) = — | coswt R(t) dt =— | cos@t R(t) dt. (2.54) 
20 a 
ae 0 
s(@) is an even function, and 
S(—@) = s(@). (2.55) 
Therefore, 
R@)= | coswt s(w) dw = 2 | coswt s(@) do. (2.56) 
9 0 
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2.4.3 Spectrum of Discrete Parameter Process, Aliasing 


For discrete process {X;}, 


1. R(r) are defined only for integer values of r, ry = 0, +1,+2,.... 


Therefore 


sw) == FIO) 


r=-@ 
2 [oe] 
Onell 
=— +— R(r) cos ro. DST 
= SRO) (2.57) 


r=1 


Here Ar, the time interval of the discrete process, is assumed to be Ar = 1. If 
Ar = 1, changew into w' =w/Ar; then 


s(w') = s(w) At. (2.58) 
From now on, in this lecture note, At = 1 is assumed. 


2. s(@) are continuous functions but are defined only for frequency —z <= 
@ = x, and are affected by aliasing. 


Roy = |e ave) r= 0, (file 2g 


(2n+1) ae EA 
= \, | e"dvw)= > | exp [i(w + 2nx)r] dV(w + 2nz) 
nae (2n-1)x ee ge 


n=—@& 


ll 
Mis 
48 


c/4 co 
exp (iwr) dV(w +2nx) = | ee” S$” dV + 2nn) = | e” dS(w) 
i —-o 


(2.59) 


Thus 


S@)= > Vw +2nn), (2.60) 


n=—o 
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and 


S(@) = a“ = ay v(@ + 27tn). (2.61) 


n=—© 


As is shown in Fig. 2.7, at frequency wo, the spectra not only of v(wo) but also of 
W(@o + 271), V(Wo + 47), V(Wo £ 62)... .. are folded as the spectrum s(wo), as 


S(W0) = V(W0) + (Mo + 27) + V(Wo + 42) + VM + O2)+.... 


= V(Wo) + V2 + Wo) + V(40 + Wo) +V(6% + W)+..... (2.61’) 


w(wo) + W227 Wo) + v(27 + Wo) t---- 


Fig. 2.7. Aliasing of the spectrum. 


This is also illustrated in Fig. 2.8, where the harmonic functions of not only frequency 
1/5, but also of 4/5, 6/5; 9/5, 11/5; 14/5, 16/5;.... can pass through the same 
sampling point O, as marked in the figure, at Ar = 1 sec intervals (see Table 2.1). 
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SEC. 


5 SEC. 


RASC CALA DEANS 
TRE ATA ER 


Fig. 2.8. Aliasing harmonic curves of frequency 1/5, 4/5, 6/5; 
9/5, 11/5; .. . can pass through the same 
sampling points at At = 1. 


Table 2.1. Aliasing frequencies. 


Equation 2.61 or 2.61” shows that, in cases of discrete process analysis of the 
spectrum ordinate at frequency w,, the power at higher frequencies 27 + wp, 


AG GES (a on doo\ote is spuriously folded on the real power at w, as in Fig. 2.7. 


Accordingly, when a continuous process is to be transformed into a discrete process 
sampled at time intervals Ar, aliasing is the most important effect to be considered. The 


analyzed range of the frequency spectrum is between the Nyquist frequencies 


-= and = (when Ar = 1, as in preceding sections —z to m ). Accordingly, 


At should be small enough to avoid aliasing. Practically it is advisable to take as 


large as = >(1.2 to 1.5)w., where w, is the uppermost frequency of the interested 


frequency component of the spectrum. Then 


22 


perenne ni Sap (2.62) 
(1.2 to 1.5)w-. 


2.5 COMPUTATION OF THE SPECTRUM FOR A DISCRETE PROCESS 

Here the procedure for calculating the spectrum for a discrete process is summa- 
rized. In many steps of the computations, the finiteness of the sample process, the 
discreteness of the data, and the choice of parameters will affect the results statistically 
and mathematically. 
25.1 Spectrum Computation Through Periodogram 

First, we take the periodogram Py(w) that is the square of the finite Fourier trans- 
form F,(@) of a single realization of this discrete process. 


N 
1 
F,(w) = —— > X, ™, (2.63) 
/2nN 2 : 
1 N 2 
Pn(@) = IF(w)\ — Sse eiwt 
ml t=1 
eel 
a SN, S Xn, X;, cos (t)—t2)o, (2.64) 
4=1 H=1 
setting 
Garat 
t2 > t+r 
1 N-1 1 
am — X, Xiir| COS rw. (2.65) 
2n r=—(N-1) N 


1 
From Eq. 2.24, we know that (=) -X;, Xt4i| 1S the biased estimate of the autocorrelation 


R(r). Therefore, 
1 N-1 


Py@)=— >. RO) cos rw. (2.66) 
r=—(N-1) 
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For the entire discrete process, 


N-1 
1 A 
E [Py@)]}=— > E [R(r)] cos ro, (2.67) 
27 
r=—(N-1) 
which, from Eq. 2.24 
N-1 
1 Ir| 
=— y (ese R(r) cos ro. (2.68) 
2m, (W-1) 


From these relationships, the spectrum computation from the periodogram is the 
same as the spectrum estimation from auto correlation. The Fast Fourier Transform gives 
the periodogram directly through direct Fourier transfer and is statistically the same as 
this periodogram analysis. Accordingly, care should be taken in selecting the spectrum 
windows to be used in the F-F.T. as will be discussed later. 


From Eq. 2.68 if N becomes ©, surely 


E [Pn(@)] > s(@) . (2.69) 


Then, from the periodogram, we can estimate the spectrum. Py(w) is an unbiased 
estimate of s(w). Here, however, R(r) is the theoretical auto correlation, and we think the 
spectrum is continuous and 


‘4 
Rr) = | so) cosrwdw , r=0,+1,+2,... (2.70) 
10 
Thus 
ge ae nlf 
E [Pa(w)] = — 1-— | suo cos ru du cos rw 
21 N 
r=—(N-1) 
—I 
rs 
tie ani 1 
= fis(i) = oe [N—tri}—{cos ru +) +cos r(u—w)} du. (2.71) 
2 N 2x SEI) 2 


Here we use the following relations on summations of digital quantities, which can 
easily be proved, 
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N-1 N-1 u 
SY CeDiO=s > > 10 (2.72) 
r=—(N-1) u=0 r=—u 
sin} (u ++ 
S yi ( hg (2.73) 
cosrp = we 
r=—u sin(i) 
a sin{(u+4)o} sin?( %¢) 
EN ea a eae (2.74) 
u=0 sin(+¢) sin’(#) 
Then from these relations, 
N-1 N-1 
ats y 1 pi cosrp@ = as » {N- Ir} cosrp 
IE TN Ny N27 r=—(N-1) 
N-1 1 sin 44) 
—_~ = F.,(¢). (2.75) 


This function is called the Fejer function F.,(@) and has the form shown in Fig. 2.9. 


NO 
When or ee 6, 


1 sin* @ N { sin@ 
F..(), 2s eee 
v@) near @ OaN sin?($) 3 ( 6 
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y\2 


4n 
N 


Fig. 2.9. Fejer Kernel Function Fepy(®) 


Therefore, 


N 
lim F =— lim 
¢—0 en) 27 6-0 


Putting this function F,,(@) into Eq. 2.71 gives 


| 


sin 0 
6 


ce 
ia 


1 
E [Py(w)] = | s)5 [Feld +w)+Fe(p—w)} dp. 


Taking into account the form of the Fejer kernel Fy gives 


= | s@F.6-w) do . 


When N—> ~, 


Fp) > 0), 


F,,, tends to Dirac’s delta function, and 
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(2.76) 


(2.77) 


(2.78) 


(2.79) 


N—o 


E [Pn(@)] = [so O0@-w) dp = sa). (2.80) 


When N is finite, E[Py(w)] is a kind of filtered spectrum, filtered by the Fejer kernel 
function F,,. 


Though the details of manipulation by statistical mathematics are not shown here, 
we know that the variance of the periodogram can be calculated as 


var. [Py(w)] is on the order of Sp), (2.81) 


27, N 
Oh Sa aL De Ue 


and also that Py(w) follow 1G , X° distribution with degree of freedom v = 2. As the 


mean and the variance of x arev and 2y (here 2 and 4, respectively), then the signal to 
noise ratio S/N is 


mean 2 


— ratio = = il. 2.82 
N ( ) 


standard variation _ (2 X 2)i/ 2 


Equations 2.80 and 2.81 can also be used to check Eq. 2.82. This S/N ratio of 1 means 
that this value of Py(@) is a very poor estimate of the spectrum. Further, the fact that 


var. [Py(@)] does not tend to 0, even when N tends to ©, means this estimate of 
E[Pn(@)] is not a consistent estimate of s(w). 

Furthermore, at two fixed neighboring frequencies w, and@ 2, cov [Py(w), 
Py(W2)] can be calculated to decrease by the increase in N. 


These facts are reflected in the erratic and widely fluctuating form of Py(w). 
Py(@) may produce a spurious peak in the region in which Py(w) is large, as shown in 
Fig. 2.10. 


25.2 Consistent Estimation of the Spectrum 
In the expression of the periodogram, 
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EjPN(w)] 
S(w) 


S(w) 
ve E[PN(w)] 


—— WwW 
Fig. 2.10. Behavior of E[Py(a)] . 
, Re, 1 N-1 
Pw) =— >  R(r) cos rw =—4RO) +2) Rr) cos rw}, (2.83) 
Dye 2m re 
r=—(N-1) r=1 
A = 1 
var. [R(r)] is on the order of O a 
and var. [Pa({q@)] is on the order of O(1) (2.84) 


are known. The reason for the large var. [Py(@)] is, as shown in Eq. 2.83, that Py(w) 
adds too many terms in TR. These terms are slightly correlated, but the basic effect of 
too many terms of YR) in Eq. 2.83 remains the same. 


Accordingly, the way to reduce this large variance in Py(w) is to reduce the number 
of additions from N to M and omit the term N > M in Eq. 2.83, 


M 
; 1 R 
E [So(@)] =— >. E [R()] cosra. (2.85) 
2 
r=-M 
This reduction decreases the variance, and from Eq. 2.84 intuitively we find 
var. [s,(@)] is on the order of 0(M/N). This can be proved by statistical mathematics, 
although the manipulation is not shown here. Substituting Eq. 2.24 into Eq. 2.85 gives 
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E [So(@)] 


I 
a= 
MIs 


E [R(r)] COs Tw 


1. Ir 
= » 1—— | R(r) cosrw > s(@), (2.86) 
hy ge N 


as M tends to > ©, but more slowly than N tends to > ©. Namely, N= ~, M—> 


but Ww — o also, for example as M was on the order of JN. 


When using the function wo(r), Eq. 2.85 is the same as, 


(N-1) 
Sow) = — yS wo(r) R(r) cosr@ (2.87) 
27 
r=—(N-1) 
where 
ces) 1 -N<-MsrsM<N (2.88) 
0 otherwise 


More generally, many forms of functions w(r) besides Eq. 2.88 are proposed as 


shown in Fig. 2.11. R(r) cos rw are even functions of r; therefore, if we take w(r) as 
real even functions of r, then 


i 1 N=1 ; 
§@) =— => wer) RQ) er”. (2.89) 
2m r=-(N-1) 
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Function w(r) is referred to as the lag window. 


Fig. 2.11. Lag windows w,(/ with A(n. 


25.3 Spectral Windows 


From Eq. 2.89, 
Net , 
§(@) = Se wOiR@ ene: 
r=—(N-1) 


As Eq. 2.66 in Section 2.5.1 shows, 


aw 
1 Nel ae. Hunde ; 
Pyw) = = RO es RG) — | Pre"? do, 
1 


r=—(N-1) 


sO inserting these relations into Eq. 2.89 gives 


a 
S@) = | Py) Ww-$¢) do, (2.90) 
IT 
where 
1 N-1 
W@)=— > wr) et. (2.91) 
2m r=—-(N-1) 


$(w) is obtained as a weighted integral of the periodogram, and weighting involves 
a smoothing operation in the neighborhood of @. This operation reduces the contribution 
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from the “tail” of autocovariance R(n), which shows more erratic fluctuations than the 
Teal value of R(r) and is Statistically less reliable, as discussed at the end of Section 2.3. 


As the inverse Fourier transfer of Eq. 2.91 


oh 
w(r) = | e’? W@) dp (2.92) 
I 
PaW, esl, s2 2 coe 28 (NTs [w(r) = 0 at Irl>N-1). 
Since w(r) are real and even, 
1 N-1 
W@)=— Ss w(r) cos ro. (2.93) 
21 r=-(N-1) 


W(@), which is the Fourier pair with w(r), is called the spectral window function, and 
w(r) the lag window function. 


wo(r), given by Eq. 2.88, is one of the lag windows and is called a “truncated 
window” or “do nothing window.” Its Fourier transform Wo(@) is from Eq. 2.73 


M 
Wo(@) = = cose = (2.94) 
—M 


This function, shown in Fig. 2.12, is also referred to as Dirichlet’s kernel function 
Dy(@). It has a high peak at @ = 0 and rather deep valleys on both sides of the main 
lobe. This sometimes causes harm to the computation of the spectrum and results in 
some spurious negative values for the spectrum ordinates. Many studies have been made 
to obtain good windows, and many different windows with different characteristics have 
been proposed and claimed to be good from different points of view. 

Several windows described by Priestley? are summarized in Fig. 2.13 and Fig. 2.14 
in the form of pairs of lag window w(r) and spectral window W(@). 


25.4 Effect of Windows 


Some of the proposed window pairs are compared in Fig. 2.15. They were designed 
mostly under the following conditions: 


1. Wy(P) = 0 is desirable to avoid the effect of large negative lobes. 
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Fig. 2.12. The Dirichlet Kernel Function [wo(o) = Dyy(6)]. 


2. | Wn(%) dp =1,1.e., w(0) = 1 from Eq. 2.92, to keep R(0) unchanged. 


3. | Wi(@) dp < ©, to make Sw) a consistent estimator. 


4s L——.. »» 


4. Wn?) > 0, uniformly as N > ~, 


Namo 


Irt\ 2 
= |w (r) 
5. Ee — 0 as N— ©, to ensure that Wy(@) is not too narrow in relation 
> wr 


1 i i : ; 
to (=) and to have s(w) an asymptotically unbiased estimator of s(w). 


The effects of the spectral windows, most of them designed under the above 
conditions, can be summarized as follows: 


1. Bias, that was 0 for the estimator “‘Periodogram Py(w),” is now 
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WINDOWS - 1 


TRUNCATED PERIODOGRAM W ($) = 


—)1----Il <u 
w(t) = fe rl >M 1 eae 


an sin ($/2) 


1 {sin Be i 
My Oar oral ein (4/2) 


DANIELL’S 


w(r) = 


GENERAL TUKEY 


1 —2a+2a cos (7r/M) 
won| 


0 


Fig. 2.13. Various window pairs (1). 
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WINDOWS - II 
TUKEY — HAMMING 
W ($) = 0.23 Du(?— 45 }+o54 Dy (4) + 


0.23 D cs 
um 02004(0 +g) 
On 


W (4) = 0.25 Dy (e- wos D,() + 0.25 Du(® + 


Mi a) 
2n M 


PARZEN 


1 —6(r/M)? + 6(|r|/M)> |r| < W (4) ~ —3_ | Sin (Mg/4)_ {4 
wioef (r/M)2 + 6(\r/M)> Ir] < M2 ~~, {5 “ | 


BARTLETT - PRIESTLEY 


aM fy _ (MBVT ig < 
st SETA sly (ME) }--1#1< nm 


w(r)= a ai |¢| > 7/M 


Fig. 2.14. Various window pairs (Il). 
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ik TRUNCATED PERIODOGRAM 


4n BARTLETT-PRIESTLEY 


Sn 


Fig. 2.15. Various spectral windows. 


bias [s(@)] = E {s() _ s(w)} - = s"(@) | gw) dw. (2.95) 
-—IT 


Here s’’(w) is the second derivative, and this is the bias at the peak of the spectrum when 
s'(w) = 0. 


2. Variance, that was s*(w) for the estimator “Periodogram Py (w)” as shown in Eq. 
2.81, becomes 


r=0 


‘ { 1 aul 
var. [s(w)] — ~ 2) | -(u)da = yo) wr, (2.96) 


where 


kG) = (=) Suh, neces. (2.97) 
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When w(r) can be expressed this way, the window is called a parameter window. 


3. Equivalent degree of freedom y, as the y° distribution that was v = 2 for the 


estimator Py(q@), changes into 


2N 2N 
0) a (2.98) 


loo) 


> wr(r) M {| Pwdu 


—-o 


4. The spectrum window blurs the real spectrum, Fig. 2.16. Its extent depends on 
the window bandwidth (BW) which is defined in several different ways, although every 
definition tries to show the equivalent frequency range over which the smoothing is per- 
formed. For a single-peaked spectrum, this blur depends on the relative bandwidths of 
the spectrum and the spectrum window. Spectrum bandwidth is defined as the frequency 
range in which the spectrum shows half of the peak of the spectrum. 


fee ae /\ 
1 7a Ne 4 
w, Ws > © te —| —~ w 
2n/M 
27/M 


Fig. 2.16. Blur of the spectrum by Daniel’s window, where the 
bandwidth is 27/M. 


(_ 
S 
Ze 


The resolution will be lower as the window bandwidth becomes narrower. 


2.5.5 Expression of Confidence Interval and Precision of §(w) 


1. By x2 Approximation 
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As was mentioned in the preceding subsections, s(w)/s(w) can be approximated by 


Ga distribution, with degree of freedom. Equivalent degree of freedom v is expressed 
by Eq. 2.98 when the window is of the parameter type. 


a 
Accordingly, ifa,(a) and b,(a) show the lower and upper 100( £) percentage 


points of ¥’ distribution, as shown in the key in Fig. 2.17, 


ie 
Prob. [y2 < a,(a)] = Prob. [y? = b,(a)] = =, (2.99) 
Probe (ay(@) ie ei (2.100) 
S(@) 
Therefore,a % confidence interval is 
Vv Vv 
S@) , —— S@) |. (2.101) 
b,(@) a,(@) 


This range is shown in Fig. 2.17 as a function of v, witha, the confidence level, of 
80%, 90%, or 95% as a parameter. 

2. Approximation by Normal Distribution; see Fig. 2.18. 

We know that when N tends to infinity, R(r) follows the normal distribution, and as 


$(@) is a linear combination of R(r), 8(@) also tends to follow the normal distribution. 
This relation can be assumed also from the fact that, when degrees of freedom vy > 00, 


as N— 0, x2—distribution tends to the normal distribution. Then, using the normal 
distribution, we can get the other expression for the confidence interval, as 


S(w) + c(x)V var. S(@). (2.102) 


Equations 2.100 and 2.101 give the confidence interval as, 


A 


1 1 A 
—— Ss@), —— S@) (2.102) 
1+ c(a) V2 1—c(a) V2 
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v/by(a) 


v/ay(a) 


CONFIDENCE BAND OF ,?-DIST. 


95% 
90% 
80% 


5 (1-2) 


Ay () by (a) 


DEG. OF FREEDOM v 
eS 


Fig. 2.17. Confidence band by y*-distribution. 


This interval is shown in Fig. 2.19 at a confidence level of 90% (c = 1.64) as an 
example, with values of the y*—distribution at the 90% level. Asv increases, the confi- 


dence band shows almost the same level as the one from the y7—distribution. 


3. Expressions of Precision of the Spectrum Estimation 


There are several ways to express the measure of precision of the spectral estimates, 
and they are related to each other as shown in Priestly.” 
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Fig. 2.18. Approximation by normal distribution. 


i.) p% Gaussian range of percentage error 0(w) 


BOS aie A. 


5) a) (2.103) 


1/2. 


where v(w) = [var. {s(@)}] : standard deviation 


bw) = E{s(w)|— s(w): bias 
c(p) is the two-sided p% point of the standardized normal distribution, 


< se) Se (2.104) 


Is(w) — s(w)I 
100 


S(@) 
ii.) The mean square percentage error 7(w) 


[zs — s(w F] _ V?@)+P?@) 


= 2.105 
2) s*(w) ae 


n°(w) = 


By Chebycheff’s inequality, 
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( CONFIDENCE BAND BY y°-DIST. — v/ay(a) ~ v/by(a) 
90% 


CONFIDENCE BAND BY N-DIST. { 1+1.64V2W } 1 ~ 
fae | 


DEG. OF FREEDOM v 


200 300 500 


Fig. 2.19. Confidence band by y2-distribution, and 
Gaussian distribution (90% level). 


: 2 
S@)—s@)I a) at) (2.106) 
| s(@) 7 a) ai: {| 


in particular if we ignore the bias 


b(w) = 0, 6@) = c(p)n@). (2.107) 


iii.) The signal to noise ratio introduced by Parzen” is 
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E(s()| (2.108) 


SNR|S(w)} = —————,. 
[var. [sw)}]/° 


If we use the asymptotic normality of $(w), for large N 


§(@)-E\S@)}| _ 5} __P_ (2.109) 
E(s@)| ae 100 
So if we set 
var. [S(@ re 
6@) = cp) (2.110) 


S(@) 


and omit the bias bw) in Eq. 2.103, i.e., if bw) =0 or E[s(w)] — s(@), then 


1 
PDE aah IS DT (2.111) 
OICe) SNRS@)| 


This relation shows that SNR {sw} is a simplified version of d(w), when the bias 


of S() is ignored. 
Also SNR can be expressed by equivalent degree of freedom, defined by Eq. 2.98 


as Vv = ———, and as S(w) follows the y7—distribution with degree of freedom 
r 


Ty 


v, where the mean and variance are v and 29, 


2v 2 
or 
v = 2[SNR{5()}]*. (2.112) 
From Eqs. 2.111 and 2.112, 
6) = c(p) /2/v. (2.113) 
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The relationship shows that, if we assume the asymptotic normality of the spectrum 
estimate, then the proportional error 6(w) can be estimated from the equivalent number of 


degrees of freedom of the spectrum estimation v, by Eq. 2.113, p and c(p) being the level 
of confidence and p/100 the point of normal distribution as 


c(@) 2 


| wieder (2.114) 
—c(@) 
From Eq. 2.98 or Table 2.2, we know that v is proportional to N/M, and for most of 
the windows 2 to 3 times N/M. If, for example, N/M = 10 to 15, then v is on the order 


of v = 30. If we adopt the confidence level of p = 0.95 (95%), then c(p) = 1.96. From 
the appropriate normal distribution 


6(w) = 1.96 2/30 = 0.506. 


This value means that the estimate of the spectrum ordinate has a proportional error in the 
order of 50% for this example. 


25.6 Choice of the Spectral Window 


It is now clear that, in order to get a consistent estimate of s(w), we have to use a 


spectral window. Several spectral windows and their effects on the estimation are summa- 
rized in Table 2.2, from Priestley.”* 
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Table 2.2. Effect of spectral windows. 
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BARTLETT 
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! 
| 
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—————— ————— 
Ee 
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ow 
& 
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The spectral windows that make the spectral estimate consistent also affect the 
estimate in several other ways, as was summarized in Section 2.5.4. The most important 
parameter of this window is the size of M, which determines the window bandwidth 
(BW), that is by Priestley~> expressed as, 


By =—— {uo}, (2.115) 
M 
where 
Dexia 1] SEE) ||. (2.116) 
u->0 lulg 
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q is the largest integer that is a characteristic exponent of the function 


k(u) = (=) = wy(r), and the value of q is listed in Table 2.2 for each representative 


spectrum window. When gq = 2, as is the case for most of the listed spectral windows, 
there is a relationship 


e/a 
b(w) is on the order of 55") | ¢°Woh)db 


1 
is on the order of aye | ex@do, @alay7) 


where 1 
K@) == | k(uje@"du 


—-c 


or k(u) = | K(@)e?“do. 


To state the relative bandwidth of the spectral windows, we must define the bandwidth B, 
of the spectrum itself. Usually, B; = w2—w, where w2andq@, are the frequency points at 
which the spectrum shows half the maximum power of the spectrum peak as in Fig. 2.20. 


Fig. 2.20. Spectrum bandwidth. 


To keep the blurring effect of the spectral window small, the bandwidth B,, of the 


Lee 
window must be small compared with the spectrum bandwidth B,. By, = mes is a popular 


guide. 
As explained in Sections 2.5.2—2.5.5 the following tendencies, as shown in Table 
2.3, are now Clear. 


Table 2.3. Statistical effects of the change of M. 


From sampling theorem it is known that, if [s(w)] is calculated from the frequency 
TON . : 4 Rn . : 
interval Aw = Ww 5(@) is completely determined, although since s(w) is a continuous 
function, it would be quite in order to evaluate it over a much finer set of points. It has 


1 
been shown that we can generally use the value Aw = 3 Bw for Daniel’s window, where 


27 
By, = Wr accordingly Aw = va When we use Aw = va the spectrum window effect 


will be replaced by a weighted mean of the ordinates, 


M 
S(w) = » aiPy(o +i). (2.118) 
i=-M 


Values of a; for several windows are shown in Table 2.4. 

This table also gives values for the coefficient proposed by H. Akaike,?> for 
minimizing the bias and variance of the estimate for the spectrum window. W> or its 
simplified form Q is generally used in most of our work, although W3 would give a 
better spectrum with steeper peaks and deeper valleys. As a general procedure for 
choosing the best of W; to W3, Akaike advises trying them all and, if no improvement 
in spectral form is recognized, adopting the window of the lower order. 
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Table 2.4. Coefficients o; for various windows. 


ee 
nies 683 
0.25 ae 0.25 
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25.7 Use of the Fast Fourier Transformation (F F.T.) Method 


There are misunderstandings sometimes that spectrum analysis by the Fast Fourier 
Transformation (F.F.T.) technique is completely different from getting the spectrum from 
analysis of auto correlation functions. However, Eq. 2.64 shows that this method is mere- 
ly the one that gives the spectrum through a periodogram, and Eq. 2.66 shows that it is 
the same as calculating the spectrum from the Fourier transform of the auto correlation 
function. 


Through the F.F.T. we get the spectrum ordinates at frequency = , or in the range 
. 2p Nig : 
-% = @ <1, at the frequency points Gee jo) 6s hes ys 65 ES Ao the dis- 


N 
crete case. This procedure corresponds to getting ms +1 ordinates in the frequency range 
0 toz, or N+1 ordinates in the frequency range —z to z. It is the same as using 


N 
ao for a rectangular spectrum window and as a result, the individual spectrum ordi- 


N 
nates of a +1 are so unreliable, statistically, and the variance so large that SNR = 100%, 


equivalent to a degree of freedom v = 2. Accordingly, the same considerations on the 
use of the spectral window as were necessary for auto correlation methods are necessary 
for the F.F.T. method after getting the “raw” ordinates at N+ 1 frequency points. Com- 
mercial programs or even a specialized spectrum analyzer through F.F.T. are now 
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available, but sometimes they do not say anything about “windows.” We have to be care- 

ful in our choice of the window to be used in the analysis to obtain a reliable spectrum. 
We can cut the number of computations from the order of N° for autocorrelation 

methods, N being the number of data, to N(r; +r2+... rg) for the F.F.T. method when 


N is factored toN =7r,-r2-... . 7g. Very commonly, N = 2? is used and then the num- 
g: Very y 


ber of computations is on the order of 2pN for F.E.T. For example, when N = 1024 = 2!°, 
the number of computations is reduced from the order of 1,000,000 to 20,000, or about 
1/50. 

The spectrum function is a powerful expression for showing the characteristics of a 
time series. However, we sometimes need the autocorrelation function to find the proper 
pair of lag and spectral windows, to decide the size of M compared with N, or to find an 
adequate statistical model from which we can go to the parametric analysis of the pro- 
cess, as will be mentioned in detail in Part II, Chapter 5. 


The autocorrelation function R(s) can be obtained as the Fourier transform of 
s(@) or as the periodogram Py(w) where, from Eq. 2.90, 


S(@) = | Pn) Wn(w —-¢) do. 


I- 


From Egs. 2.64, 2.63, and 2.66, 


Py(wp) = |Fx(wp)l’, (2.64’) 
ih pee N 
F,(p) Tai X, eM, p=0,1, ... > (2.63”) 
t=1 
1 N-1 . 
Py@,)=— > Rr) er’, : (2.66’) 
r=—(N-1) 


N 
Analogously, from Eq. 2.64’ or from Eq. 2.66’, we see that He ordinates of Py(w,) it 


is impossible to get N values of R(r),r =1, ...N. We get only R(r) values at r < ~. 


In order to get R(r) values at r = 0 toN, we need N ordinates of the spectrum or the 
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periodogram Py(w,). To make this possible we need N more data, and that can be real- 
ized by adding N 0’s to the original data,as x» ..... SHEN O,O,0, oo 00 We 


N N 


2N 


In this way the FFT. method can also be used for computing the correlation function 
from the spectrum ordinates at N frequency points as in Fig. 2.21. 


I 

| 

N 
FREQ. POINTS | 


FREQ. POINTS 


Fig. 2.21. Frequencies to calculate the spectrum. 


25.8 Filtering 


As was mentioned in Section 2.4.3, in sampling a continuous time series for compu- 
tation of the spectrum, we have to pay attention to aliasing, and the sampling interval Ar 
should be small enough to avoid the aliasing. We must also be careful about the leakage 
of power or the blurring effect through the spectral window, especially when the spectrum 
has sharp peaks or steep valleys, because the spectral window acts as a smoothing filter. 
Besides, sometimes we are especially interested in the spectrum over a certain range of 
frequencies. Then the filtering technique is helpful in dividing the power of the spectrum 
by the frequencies or in modifying the shape of the spectrum to a shape more easily han- 
died. 


Generally, for a continuous process, the filtering effect can be expressed as 


Y(t) = | g(t)X(t —T)dr, (2.119) 


where X(t), Y(t) are the original and filtered processes, respectively, and g(t) shows the 
filtering effect in the time domain. For a physically realizable filter g(t) = 0, for t < 0, the 
range of integral can be from 0 to ©. Then from the discussion in the preceding 
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subsections and assuming the existence of a spectrum of X(t), Y(r) and a Fourier trans- 


form of g(u), 
Syy(@) = |IG@)?sxx(w), (2.120) 
G(w) = | g(t)e "dr. (2.121) 


A few samples of simple and ideal filters are given as follows: 
1. Band—pass filter (Fig. 2.22) 


G (w) 


Wo -w, (0) w, OW — @® 


Fig. 2.22. Band—pass filter. 


1 ; fora, < lol < a2 


IG@)? = (2.122) 
otherwise 
Then 
lee) @2 
g(t) = uN G(w)e'dw = as | eda 
2 2 
—-3o @, 


SiN WoT — SiIN@ T 
= oe ee) forall’r. (2.123) 
aT 


From the form of this function and Eq. 2.119, we need the input process X(t) for 
— © <t< ©, so theoretically this filter is physically unrealizable. 
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2. Low-pass filter (Fig. 2.23) 


Fig. 2.23. Low-—pass filter. 


When w, = 0 for a band-pass filter, 


IG@)? = (2.124) 
0 lal > wo 
g(t) = a (2.125) 


3. High-—pass filter (Fig. 2.24) 


Wo 0 Wo —-> 


Fig. 2.24. High—pass filter. 


When w,— © for a band-pass filter, 


0 lal S wo 


IG@)!? = (2.126) 


1 lol > wo. 


Also for digital processes, we can express the digital filtering by 
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k 
Y,= ») Boi (2.127) 
tT=0 


syy(@) = |G@)P’sxx(w) (2.128) 
k . 
G(@) = Ss &r ene, (2.129) 
tT=0 


4. Low-pass digital filter (Fig. 2.25) 


{ G (w) 
1.0 


71 — @ 1 
Fig. 2.25. Low-—pass filter. 


For example, the simplest equation for this type will be 


1 
Y; = race + X;). (2.130) 


Then 
syy(w) = IG(w)?sxx(w) 


1 2 
= a +e! | syx(w) 


=5(1 + cos w}syx(w). (2.131) 


5. High-pass digital filter (Fig. 2.26) 
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Fig. 2.26. High—pass filter. 
The simplest equation for this type will be 


Y,= 5X1 —X,) (2.132) 
Then 
1 2 
S| [el —e 1 | syxw) 
1 
= ail —cos w}sxx(w). (2.133) 


6. Pre—whitening digital filter (Fig. 2.27) 


Sxx(w) 
Syy(w) 


-T 0 Tl 


Fig. 2.27. Pre—whitening filter |syy{@)/Sxx(@) |". 


If a digital filter such as 


Y,; = QoX, + Q1Xj,-1 + Q2X-2 » . . ArXik.--, (2.134) 
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is used and the spectrum syy(w) becomes a white specttum syy(w) = 0}, then as will be 
shown in Section 5.4.2 of Part II, 


Syy(@) = a; = lap + aye + ane +... + ape syy(w). (2.135) 
Then 
o} 
Sxx(@) = - —. (2.136) 
xn lag + aye7? + age +... tape HO? 


If we can find the filter that will make this spectrum completely white, that filter is 
called a complete pre—whitening filter. Then from Eq. 2.136, we can find syy(w) from 


the variance o¥ of the filtered process (that is uniform), and the frequency function 


IG) = lag + aye + ane 2 +... tape. (2.137) 


This tells us that finding the pre—whitening filter G(w) is the same procedure as fit- 
ting a model to the X; process expressed as 


€;= 0X; + a4X;1+ 66.6 0 QyX 14+ 050 o>” (2.138) 


where €, is a completely random process with variance ay. This is the problem of AR— 


model fitting to the process X(t), which will be discussed in detail in Chapter 5, Part II of 
this paper. 


2.6 MULTI-VARIATE SPECTRAL ANALYSIS; SPECTRAL 
ANALYSIS OF FREQUENCY RESPONSE 


2.6.1 Two Variate Spectral Analysis 
If there are two stochastic processes {X14} {Xo 4} C= 00-2 se eeach' 
weakly stationary, then Cov. {x Lh X2..| is a function of (t; — rt) only. 


For this stationary bivariate process, the correlation matrix is defined as 


Ris(r) ~—-Ryar) 


; 2.139 
Ror) Roar) Ce 


R(r) = 


where 
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Rus(r) = E| [X14 —Ha) [Saree Ha] 

Rog(r) = E| [X21 Mo) Xou0r—H}| 

Ro,(r) = E| {X10 iol Grae - | eel a aideadiie Kon mn@-120) 
Ryo(7) = E| [X2,- ba} {X1e4r— -n}| a: Xp leading X; 


Ry(-r) = Ri) 


(2.141) 
Ry2(r) = Ra1Cr) 
*Denotes the complex conjugate. 
_ Ru) 
Q11(7) = Ri) 
R ; 
Q22(7) = == (2.142) 
R21(7) 
0218) = eer TO 
(R11(0) « Rao(0)}/? 
Then the spectrum matrix is 
bone 511M) — 512(@) (2.143) 
S21) — S22(W) 


where 
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dpe 
511(@) = — Ryne? 
1(@) ee (7) 
1 = —ir@ 
S22(@) = > d Rnlre 
PSS (2.144) 
li. se 
S21(@) = = SO Rui@er a a X leading X> 
1 rw 9 
512(M@) = = y Riper see X> leading X). 


For the spectrum to exist, the correlation function must be absolutely summable, for 
example, 


SY Rar) < &. (2.145) 


—-o 


By spectral representation, using the Fourier—Stieltjes form, 


a a 
Xir= | e"°dZ1(w) X24= | edZ>(w) (2.146) 


= ’ 
0, ete (2.147) 


BNO) CAO etn cp ceen 


Z ’ 
0, Seaman (2.148) 


EldZ3(w) dZp(w')] = 5S2(w)dw, W=ao 


dZ\(w), dZ\(w'); dZ>(w), dZ2(w') are orthogonal, respectively, and also cross orthogonal, 


Z ’ 
0, alias (2.149) 


ElaZ(@)\a2(o))— ays agen 


ap) 


0, o za’ 


E[dZ3(w) dZ\(w')] = as (2.150) 
5y2(@)dw, w=O 
Inverting Eq. 2.144 gives for the two variate process 
7 4 
Ro(r) = | e*saywrde, R12(r) = | esio(orde), (2.151) 
I I 
as was for the single variate process 
i4 P14 
Rii(r) = | e's) \(@)dw, Ror) = | e syo(w do). (2.152) 
It TT 


When X;, leads X2, Eq. 2.144 gives 


een 
SG) ime IR ems. 


r=-@ 


Here, since FR (r) is not symmetrical on r = 0, the cross spectrum 52;(m) is a complex 
function, 


= Co21(@) + iQu21(@), (2.153) 


Co21(@) = = y 5 (Rast) + Ro\(- r)} cosrao, (2.154) 
1 ed 
Qu21(@) Oe ») 5 Rai) + Ra r)} sinra. (2.155) 


Co2;(@) and Quz;(@) are the co— and quadrature—spectra of X;(t) and X(t) when 
X(t) leads X2(t). 
Thus the cross spectrum is expressed by its absolute value and argument as 
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Is21(@)| = {Co3w) + Qui(w)} (2.156) 


1] Co2(@) (2.157) 


Arg [s21(@)] = tan” Quz(w) 


and 
dolso(w)| eAtels)] = 5,,(w)dw 
= E[dZ\(w) dZ>(w)). (2.158) 


In this case the coherency function 3;(w) is defined as 


@) 21) cov|dZi(w) dZx(w)| mate 
o) = ——— =. : 
oes (s11@)s2x(w)}/?_— [var{aZ1(w)} var[dZ2(w)}}!/? 


Thus 72;(@) is the correlation coefficient of dZ}(w) and dZ>(w). Here, 


0<l2(@)l <1 at alla. (2.160) 


Usually ty2;(@)l or ty2;(w)I? is called the coherency function and shows the extent 
to which X2,t and Xj,t are linearly related. 


2.6.2 Linear Responses 
Suppose there is a linear system, X, being the input, Y, the output as in Fig. 2.28. If 


Y= >) a Xen. (2.161) 


T=-0O 


{g:| is called the impulse response function. If this is physically realizable, 


g:=0 for tT <0. 
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INPUT OUTPUT 


xe Ye 
LINEAR SYSTEM 


Fig. 2.28 Linear system. 


However, we had better treat g, for— © — + ©, because sometimes X, itself does 
not express the real cause of Y,, and g, can even be non—zero at t < 0. We assume, for the 
impulse response, 


> Igil < @. (2.162) 
Then 
Syy(@) = IG)? Sxx(@), (2.163) 
where 
G(@) = yy 81 € WT. (2.164) 
T=-0 


G(q) is called the transfer function or frequency response function of Y, to X;. For 


Syy(@) to be finite in total power | Syy(w)dw < ©, syy(w) must be a bounded function 


of w and | IG(@@) "dw < ©. Accordingly, by Parseval’s relation, 


Wigear< 00. 


Using Eq. 2.161 gives 


58 


Cc Temas | ce 


EE ee en | 


Ryx(r) = E [XY] = > 8 ELK; Xue] 


T=-@ 
= Sy &, Rxx(r—-T), (2.165) 
T=-@ 
and therefore 
1 ce 
syx(@) => > e@'Ryx(r) 


=G(@) Sxx(@). (2.166) 
Accordingly, 
Cy ae, (2.167) 
Sxx(@ ) 
From Eq. 2.163 
IG@)? = 2) 2.163” 
G@) eG) (2.163”) 


These equations show that, from the spectra of output and input, we get the IG(w)!’, the 
response amplitude function, but if we need the complete response function including the 
phase relation, we have to use the cross spectrum syx(w) as shown in Eq. 2.167. 


In this ideal linear case 


yo)? = IO) aan _ IG@)? 
| 


=——, = 1. (2.168) 
511(@) son(o)| Fecal IG(w)!* 


The coherency ty(w)/? should be 1. 


2.6.3 Linear Response in the Presence of Noise 
When noise is added to the output Y;, as in Fig. 2.29, where we assume N, is real— 
valued, zero mean, uncorrelated with Y, and with X,. 
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1G SG eINGES ot Xan ceINh | (2.169) 


—-co 


X, Y, , 
————| _ LINEAR SYSTEM YK 


Fig. 2.29. Linear system in the presence of noise. 


Then, 
Ry'x(r) = E[X# Yr] 
= > gr EIX# Xtra] + EX# Nc] i) 
T=-—0 
ad Ss &r Rr ire a) 
T=—-O 
because, 
E[X# Ni] = 0. (2.171) 
Therefore 
1 co : fo °} 
sy x) = Sa SS Bee G 2) 
r=-@ t=-@ 
= G@W)sxx(@) = Syx(@) (2.172) 
and 
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CO Sy'x(M) _ Syx{@) 


= (2.173) 
Sxx(@)  Sxx(@) 


This shows that, if we take the cross spectrum of Y,’ with X,, the frequency response 
function G(@w) can be obtained as the ratio of Syx(@) to the spectrum of input sxx(w) as 


was the ideal case in Eq. 2.167, because syx(w) is not affected by the existence of noise 
N;. However, taking the spectrum from Eq. 2.169 gives 


sy'y'(w) = IG(w)Psxx(w) + syn(@) = syy(@) + syn). 


(2.174) 
Therefore, the coherency y?(w) is 
Syx(w) 2 
52 ey ees) bea 
ay ms ~  Spy(@) 
Sra) Sox) eer 
_ {G@){syy@) — sxn@)} 
[srr(@)-syo)} 
Syxx(@) X Syry(W) 
SNN() 
= 1-——_ = 1-7). (2.175) 
Sy'y(@) 
Here 


2 __ SNN() 
4(@)° = —— 2.176 
Sy'y'(@) ( ) 
is called the residual error. 


These results show that it is essential to calculate the cross spectrum to get a good 
estimate of the frequency response function. 
2.6.4 Multiple Inputs Multiple Outputs Case 


More generally for k multiple inputs and / multiple outputs, as in Fig. 2.30, the 
output is expressed by 
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Yi.= y Sir Xie + » Gigs AOpeP 3 00 9 y Sika Xk-t-r, 


T=-@ T=-@ T=-O 
lee) 


Se he > Bik Xki-z> i=1to/. (2.177) 


T=-—0 


or in vector form, 


Y,= ay &: Xp - (2.178) 
T=-0 


[1 x 1] [1 x k] [k x 1] 


Fig. 2.30. Multiple inputs multiple outputs system. 


If the output is expressed by spectral representation, then 


aw 
Yi,= | et gw) i= 1 tol, (2.179) 
1 
where 
dZ°”(w) = G,. (@)dZP(w) +... . +G;_(@)dZP(w) i=1 tol. (2.180) 


2.6.5 Multiple Inputs Single Output Case, Multiple and Partial Coherencies 
A multiple inputs, single output case, as in Fig. 2.31, is a special case of Eq. 2.177. 
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Y,= » Sir Xie + » Boyes Gye a? 9 6 0 0 Sy Gkx Xkpr- 


T=-@ 


Thus 


and 


In the same way, 


Fig. 2.31. Multiple inputs single output system. 


T=-@ T=-O 


SY 8a 8p Ry (@-B +r) 


t=ljy=la co B 
k k 

syX(@) = > >) Gio) G@) 5; {@). 
i=1 j=1 


k 
sy¥(@) = > Gio), 5; {), 


t=1 


where R; FCs) = Rxx,7), S; j@) = Sy, x), and sy(@) = Syx(@). 


The input is 


Then setting 


X:= [Xi4 X21 cos ee Xx). 


G(w) = [Gi@), Go@) . . . G<@)], 
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(2.181) 


(2.182) 


(2.183) 


(2.184) 


(2.185) 


(2.186) 


511) sy2o@) . . Six(@) 
S21(@) S22(@)  S2x(@) 


Sxx(@) = (2.187) 
Sk1 Se2(M) . . Sx(@) 
511@) sy(@) . . 51K@) |[Gi@) 
S21@) . 5S2x(@) || Ga@) 
sy) = [G(@)G2@) . . . G(w)]] . | e188) 
Sk1@) sin(@) . . Sixx) JLGE@) 
gives 
Syy (@) = G'(@) Sxx(@) G* (@). (2.188 ”) 
Here G'(w) is the transpose of the vector G(w). 
Also 
Sy1(@) 511) sy2(w) .. . . S1x{@) Gi) 
S¥2 521@) 522) S2(W) G2(@) 
=| ° ; ; Mea (2.189) 
Sy) Sk1 SxQ(@) . . . - Sek) G,@) 
gives 
Syx(@) = Sxx(@)- G@). (2.189”) 
Therefore 
G(@) = sxx) - sxx), (2.190) 
or 


(2.190”) 


Gi(w) Sy) 


Thus the multiple coherency v}(w) is 


OE Syx (@) f Syy(@) 
Sxx (@) Sxx (@) 
_ syx(@) syx(@) Tee 


= a ee yf = : 2.191 
AROGEO) SG) © OP 22» ee 


Here, as mentioned by L. Tick,?°H. Akaike,?” and Enochsen,”* the conditional 


spectum § 31@) is defined as 


2 
512(@) 
5,3 @) = 511@) - 522) X aC 
2 Isp(@)? _ Ilsio(@)I? 
Sy acon) 
= 511(@){1-y12(@)}. (2.192) 


This shows the spectrum of X;X; under the condition that X> has occurred, that is, the 
spectrum of XX; masked the effect of Xp. 
Generally, if the conditional spectrum of syx(w), under the condition that x; has 


occurred, is expressed by sya. ,(), then the partial coherency is defined as 


os (OS (2.193) 
Sire S19... LO Syy 2.3 x) 


65 


THIS PAGE INTENTIONALLY LEFT BLANK 


66 


CHAPTER 3 
CONSIDERATIONS ON THE IMPROVEMENT OF COHERENCY FUNCTIONS 


3.1 INTRODUCTION 
Coherency functions y(w) can be considered good clues to finding the extent to 
which the system can be approximated as linear to the input X, which can also be consid- 


ered linear. If there is noise in the output Y,, as was discussed in Section 2.6.3, the 


coherency function will be reduced, usually to coherencies of less than 1. It is difficult to 
find the real reason for this reduction, although Eqs. 2.175 and 2.176 tell us it is the effect 
of noise. Here not only noise in the output but also computational error, statistical bias, 
nonlinearity of the response characteristics, and the effect of feedback are all counted as 
noise in the result. All the effects that cannot be accounted for by linear, open relations 
are counted as noise. The author has made a few suggestions for improving the nonpara- 
metric spectrum analysis and the resulting computation of coherencies. 


3.2 SHIFT OF THE OUTPUT IN CALCULATING THE CROSS SPECTRUM 


R;(r), the auto correlation function of a real process i(f), is an even function and 
shows a maximum value at r = 0, the origin of the lag r. The lag window applied to this 
auto correlation to get consistent estimates of the spectrum ordinates is also usually an 
even function and has its peak at r = 0, and w(0) = 1, as in Fig 2.11. Accordingly, R(0) 
remains unchanged and then gradually decreases toward r=+ M. This procedure keeps 
the most reliable and important part of the correlation near r = 0 almost unchanged and 
reduces the contribution of the correlation at larger values of r near + M, where the 
correlation is less reliable, to nearly zero, preventing the formation of large negative lobes 
in the spectral window. This way is reasonable to get consistent estimates of the spectral 
ordinates and make the spectral window fulfill conditions 1—5 in Section 2.5.4. 

However, in getting the cross correlation, if we use lag windows in the same way, 
sometimes important information is lost, and the result is an apparent reduction in coher- 
encies. This was studied theoretically by N. Akaike and Y. Yamanouchi”? (1962), but a 
more intuitive explanation by this author [Yamanouchi*° (1961)] is given here. 

In cross correlation R;,(r), if the phase relation of the output j(r) lags or leads the 
input i(r) considerably, the maximum value of the cross correlation, which is not the even 
function, will no longer lie at the origin r = 0 but will lie at a larger or minus value of r 
(distant from the origin). In applying lag windows, if we set the maximum of the lag win- 
dow at r = 0, sometimes important information is lost at the peak of cross correlation. A 
small value of the lag window will be multiplied to the peak value of the cross correlation 
at a larger or minus value of r, and will keep the less important part of the cross correla- 
tion almost unchanged by the windows, as shown in the example in Fig. 3.1. This is not 
reasonable, so the author suggested shifting the origin of the cross correlation to its peak 
point and then applying the lag windows. The amount of the change of phase from this 
shift ro should, of course, be used later to modify the phase relation by row. 
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Fig. 3.1. Shift of cross correlation in applying the lag window. 


This alignment technique can be expressed as a shift in lag windows as well as a 
shift in the origin of the correlation, or as a shift of the output time series relative to the 
input time series, 


re (N-1}-ro 
SAo)=— SS wr-ro)Rir)e“*”, (3.1) 
Cer ee (N-1)-ro 


where w(r) is the standard type of lag window for estimation of the auto—spectra. 
Thus 


; (N-1) 
S(O) =— > WORT + roe), (3.2) 
r=—(N-1) 


So if we set 
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1 : 
§i(@) = Dd wR Ar + roe”, (3.3) 


then 


Sj(@) = Sj(@) X gM, (3.4) 


The phase is shifted by wr by this shift in the origin. Accordingly the phase of the 
output should be modified by — row after the cross spectrum analysis. Figure 3.2 shows an 
example of this shift in the analysis of a 5—ft model ship rolling in tank waves?! (beam 
sea, without advance speed). From this figure, we find the amount of shift on cross corre- 
lation should be ro = 9. The results of the analysis are shown in Figs. 3.3 and 3.4. The 
improvement in the coherencies due to the shift is clearly shown in the upper part of Fig. 
3.4. The improvements in phase relation and in gain are also apparent, although these im- 
provements are partly due to the large m(= 60 > 40). 


x10~4 (FEET)? CORRELOGRAM ERUNING 1833 = 
15 . WAVE Qwir) ea 31SEC : 
t tT = 0. 10 


(FEET-DEG) ROLL-WAVE Qpwit) 

0.046 
t 

0.02 


Fig. 3.2. Auto and cross correlations of wave, roll, roll-wave 
for a model ship in tank (run 834). 
(From Yamanouchi.3') 


Of special interest to us is that the improvement in coherencies is also reflected in 
the change of bandwidth of the cross spectrum. Before the shift in origin, the co— and 
quad-—spectra in Fig. 3.3 had a narrower bandwidth than after the shift, showing that the 
blurring effect of the spectral window is smaller for the cross spectra with broader band- 
width than for those with narrower bandwidth. Thus this shift improved the estimate of 
the cross spectra by reducing the leakage of power. Although we refer to improvement, 
we do not have experimental data in regular waves for comparison in this run. However, 
in Figs. 3.5—3.7, which show the same kind of result for run 832 and the data in regular 
waves (Fig. 3.7), we notice the same tendency and conclude that the results obtained by 


69 


this shift are best in this analysis and closest to the experimental data obtained in regular 


waves. 


CROSS SPECTRUM 


FT.DEG.SEC 
1 
ROLL-WAVE 


x10~4 (FEET)*-SEC 
SPECTRUM RUN NO. 834 
WITH BK. 


v=0 

SPECTRUM II 
m = 40 0 
m = 60 ° 


— -e — (m= 40, WITHOUT SHIFT) 
—<o— (m = 60, WITH SHIFT 9.47) 


Fig. 3.3. Auto and cross spectra of wave, roll, rolt-wave (run 834) (max. 
lag no. m=60 with shift and m=40 without shift). 


(From Yamanouchi.3") 


3.3. IMPULSE RESPONSE FUNCTIONS FROM THE CROSS 
AND AUTO CORRELATIONS 


By Eq. 2.169 for a linear system, 


We= yy ee Xe: (3.5) 


u=—-@ 


where g,, is the linear impulse response function of the linear system, X, and Y, are the 
input and output, respectively, and N, is the noise, uncorrelated with X,and Y,. By Eq. 


2.164 
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G@)= > ge (3.6) 


u=-@ 


where G(@) is the frequency response function of this system. Then as already shown by 
Egs. 2.165, 2.170; 2.166, 2.172, 


Ryx(r) = > gy Rxx(r—u) (3.7) 


— -e — (M= 40, WITHOUT SHIFT) 
——>— (m = 60, WITH SHIFT 94) 


| | 
—_———_— 
| WITH BK. | 
|}V=0 | 


RUN NO. 834 


| 


| SPECTRUM J | 
50+ RESPONSE |Hi())/F loco son) | 
} 


] |m=60 —o- 
40 : 1 —— 
H | 


30= 


| 
| 
| 
} 


Fig. 3.4. Frequency responses of roll-wave (run 834) 
(m=60 with shift and m=40 without shift). 


(From Yamanouchi.3") 


Syx(@) = G@) Sxx(w) (3.8) 


If we assume that g, (with— © <u< ©) can be approximated by finite terms of 
8. (with—n <u<n), then Eq. 3.7 gives the matrix of Eq. 3.9, 


Tl 


CORRELOGRAM 


4 
x10~4 (FEET! | RUN NO. 832 tS 
15 WAVE Qy(T) (SPECTRUM 1) ss ie 
10 


At = 0.5 SEC. 


mf P 


Fig. 3.5. Auto and cross correlation of wave, roll, rol--wave of a 
model ship (run 832). 
(From Yamanouchi.3") 


Ryx(—n) Ryy(0)  Ryx(l)..... Rxyx(n) . . . . Ryy(2n) igus 
Ryx(—n+1)| |Rxx) —- Rxx(0)... . Rxx(n—1) . . Ryx(2n—-1)| | g-ne1 

; le : eas 
Ryx(0) Rxx(n) —s- Rxx(n—-1) . . . RxxX(0) . . . .. Rxx(n) 80 
Ryx(n— 1) Ryx(2n—- 1) Ryy(2n—2) . . Ryx(n—1) . . Rxx(1) 8n-1 
Ryx(n) Ryx(2n) — Ryy(2n—1) . . Rxx(n).. . . xx) n 


using the relation R(—r) = R(r). Here for the cross correlations, we use the shifted version 


of Ryx(r), as Ryx(0) is to be the maxima over the range r =—n to n, and for the auto cor- 
relation, the range 0 to 2n. The matrix on the right hand side of Eq. 3.9 is large on the 
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FT. DEG. SEC. CROSS SPECTRUM 


WITHOUT B.K. 


V=0 
SPECTRUM II 
m= 40---e-- 


ao (FEET)*-SEC. 
SPECTRUM AUN a CeO 


VANES WITHOUT BK. 


vV=0 
SPECTRUM II 


ONLY m = 60 WITH SHIFT 
m = 40, 120 WITH— 
OUT SHIFT 


te) 1 2 3 4 5 6 7 8 9 10 tls) 
—_— SEC 
Fig. 3.6. Auto and cross spectra of wave, roll, rol-ewave (run 832) 
(m=60 with shift, and m=40 and 120 without shift). 
(From Yamanouchi.3") 


diagonal and is symmetrical around the diagonal. The solution of Eq. 3.9 gives g,, over 


the range u =—n to n. This author [Yamanouchi*?] believes this method provides the lag 
window free estimate of the impulse response function, or the spectral window free esti- 
mate of the frequency response function from Eq. 3.6. The choice of the lag window or 
the spectrum window is a serious problem in getting a good estimation of the spectra and 
the frequency response relations of the output to the input, as was discussed in Sections 
2.5.2—2.5.6. This method eliminates the problem of windows and frees the calculation 
from their blurring effect. The use of Eq. 3.9 might eliminate also some of the uncertain- 
ties and errors in the algorithm for the correlation. 


Figures 3.8—3.11 show the analysis of the rolling of model ships afloat in irregular 
beam seas. g, for—90 to 90 was analyzed using Ryx(w) of — 90 to 90 and Ryyx of 0 to 


180. The correlations in Fig. 3.8, where the cross correlations are already shifted by 9, 
are shown in normalized form and the calculated g,, are shown at the top of Fig. 3.9 for 


u of — 30 to + 30 only, though the g,, were computed for u of — 90 to + 90. For compari- 


son, the impulse response function obtained as the Fourier inverse transform of the 
frequency response function calculated by cross and auto spectrum analysis is shown at 
the bottom of Fig. 3.9. They look very similar. 


The frequency response function obtained as the Fourier transform of the impulse 
response function g,, calculated by this method is shown in Fig. 3.10 together with the 


results of cross and auto spectral analysis. Again, the frequency response function 
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COHERENCY 


WITHOUT BLK. 


_|v=o 
SPECTRUM II 


Fig. 3.7. Frequency responses of rol--waves (run 832). 
(From Yamanouchi.3") 


IG(@)I, i.e., the amplitude gain, from these different methods is very similar. From the 
time history of inputs X, of the waves, and the use of Eq. 3.5 which neglects the noise 
N,, the output Y, was synthesized, using values of g,, u =—90 to 90, as shown in Fig. 
3.11. These results show that this method gives reasonably good results. 

This example is for the roll of a model ship where, because of the small damping of 
the motion, the impulse response function g,, is slow to decay. We need a larger number of 
terms, i.e., g, for a longer range of u. With much larger damping, as with the heave or 
pitch response of a real ship, this author believes we will get better results. 

Here the more statistical, stricter estimation of reliability or confidence in the results 
is lacking. This author found later that the procedure was the same as that to solve the 
Yule—Walker equations used for the AR—model fitting, that will be mentioned in Part II. 


Accordingly, the above mentioned belief was more strictly examined statistically as the 
choice of order in AR—model fitting, as mentioned in Section 5.5. 


3.4 EXAMPLE OF MULTIPLE INPUT ANALYSIS, A TRIAL FOR 
NONLINEAR ANALYSIS OF SHIP’S RESPONSE 


Here a multiple input analysis of a ship’s behavior at sea, performed by this author 
[Yamanouchi>}], will be shown to demonstrate the usefulness of the method. Often, one 


74 


CORRELOGRAM 


ERMIR EEA 
Sey 


-1.0 
Fig. 3.8. Auto correlation of waves, roll, and cross correlation of 


wave-—foll of a model ship (shift 7 = 9). 
(From Yamanouchi.?2) 


IMPULSE RESPONSE 0.1 
FROM Ryz(t) = A(t) * Axx(t) 
> : CONVOLUTION 0.05 


Fig. 3.9. Impulse response of roll to wave height (without advance speed). 
(From Yamanouchi.?) 


output is analyzed as the response to a single definite input. This approach is reasonable 
when the real source of excitation for the system is actually single and no other source 
need be considered, even if several outputs are correlated or combined. For example, 
even though the sway, roll and yaw, or pitch, heave and surge motions of a ship are 
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UNDERSTANDING AND PREDICTION OF SHIP MOTIONS 


RELATIVE ERROR 


1.0 . 

oe | mY ‘ j 
0.6 

0.4 ce se 

0.2 b RY Ye W/ 


balboa | bw COHERENCY 


PHASE SHIFT 


-2x 


0.8- 


0.7F AMPLITUDE 
0.6+ GAIN 


A FOURIER INVERSION OF 
IMPULSE RESPONSE 


RESPONSE OF ROLL TO WAVE HEIGHT 


Fig. 3.10. Frequency response of roll to wave height. 
(From Yamanouchi.?2) 


WAVE ELEVATION OBSERVED TIME - 


ROLL MOTION. 


ie BSERVED 8°P 
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oTIGN 8°P OBSERVED!_99 m/m_ x BY FOURIER JTRaNs OF H (w). © BY h, FROM 


Fig. 3.11. Comparison of synthesized and observed output of roll. 
(From Yamanouchi.?2) 
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correlated and combined with each other, we can get the response of a single roll, sway, 
or yaw and so on, to the wave (either slope or height), as frequency response functions by 
solving the simultaneous equations of motion of the necessary degree of freedom, consid- 
ering the waves as only one source of excitation. 


This is not the case, however, when one output must be considered as the response 
to many related inputs. Shown here is an example that this author encountered in the 
analysis of a ship’s motion and stress. The ship was a cargo liner on the New York line, 
under service in winter on the North Pacific Ocean.*4 The relative wave heights of en- 
countering waves were measured by an on-board ultrasonic sensor located on the side of 
the hull near midship. Because of the poor position of the measuring device, no attempt 
was made to convert the readings to absolute encountering wave heights. However, the 
real encountering waves are only one input to the stresses and the motions of the ship. In 
analyzing the transverse stresses induced on the web frame near midship, this author tried 
to express the effect of real waves by the relative wave heights measured at the side of the 
ship and by supplementing with other inputs like rolling, pitching, and vertical accelera- 
tion measured at the same time. These additional inputs were correlated with each other, 
as shown in Fig. 3.12, and multiple input analysis techniques were adopted to analyze the 
response. 
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{ X9 Yo y 
PITCHING ————_—_——> ©) STRESS 


x 


{ 3 Y¥3 
OTHER " WAVE HEIGHT 


EFFECT: 


HEAVING 


Fig. 3.12. Multiple inputs interpretation. 
(From Yamanouchi.?3) 


With stress as the output and such inputs as relative wave height, rolling, pitching, 
and vertical acceleration, and with various numbers of inputs, the effects of some particu- 
lar inputs were investigated by checking the multiple and partial coherencies that show 
the effects of a particular input in the presence of one or more other inputs. 


Another trial investigation was the analysis of nonlinearity of output to input. Con- 
siderations on nonlinearity that provide another reason for getting a poor coherency 
function are generally treated in Part II of this lecture, but for convenience one trial is 
described here in relation to multiple input analysis. Figure 3.13 is an example of a record 
of simultaneous measurement of many responses of this ship such as roll, relative wave 
height, vertical accelerations at four points, stress on the web frame, revolution and 
torque of the propeller, helm angle, yaw, encounter time of waves to the stern, and so on. 
Some of the responses were picked up, sampled, digitized, and punched on a tape after 
the test. (At that time data were processed using punched tapes.) 


77 


(g¢!4yonouewe, wold) 


‘(022 ‘Ou }s9}) 4aps0d-IS1A UO Psoo—as jo sajdwuexy ‘ete ‘Bi4 


oas | 
= es a r ~— + = ——F Tis T 20v4yNS 
sooo s v i] M Ss v 


YW M Ss Vv ts] M Ss Vv ts} M Ss Vv is) M Ss Vv t=] M Ss Vv i=] M Ss v YSAVM SSau1S 90v 110H 
bpO- b0Z+ 020+ 661- SLO- 16€+ 00+ 9pz+ 100- cll~ ClO+ 296+ 640+ GEE- 620- Lez+ Z90-- 110+ 120+ G@2- Evl- 1BE+ BEOt 92E+ GSO- Bli+ Ihl+ gap+ OLO+ 1S2- 621+ 1Sh+ O00- OSz- 60+ 


oO o oO ° oo 00 fe) ° {e) ° oO oO ¢ Oo 000 Oo 
Qo oo 00 00 9 000 o oo °o ov 00 600 B) ° ooo o 90 ° ° oc 00 ° ° 000000 
° oo i) ° ° ta) ° © 0 oO o 00 00 00 9 
©28000970000e7200n 0AM GOH nH aM OM ANODE HAA orn eroanneeconco20000000000eN20082000n0000001 100nn oe oCeD HO DDGCO CON 
90 9 0 000 9 0.0 06 0 0 oO 900 ° 9 oo 900 900 00000 9 000 ° 
2) 090 0 00 90 0 oO 9000 0 © 0 oo Oo 09 0 00000 ° i) 900 0 900 ° ° 
©0 000 Oo oO 00 09 oO 00 oo 0 oO ©0900 0 © 00 on oo 0 00 © 0 oo ° 
HAZILISIG JO 3dVL GAHONNd NO GHOd3HY 


(Liv) 99V,5 S aval 


OT TO 
m wd Maat? NMoG 


(moa) sov\nl | 


(3dAL NOILONGND NOILNIOASY f 


Ye el a eee eet Ps es a aaa al alata taal otal eat ate Lette! 
a = S S ea (co fad Paes fe ee | Sel is = aa] foe =. 1 if ca isn Vom Viens Vesa ape Rar Sak fi 
oe = Piet tt ee er SEV (I) 43QHOO-ISIA NO GYOOSY 


Maas a a ct a Wooo ani a a ET rere rere; tht ir 
Y31LNNOONS 03S 1) SWiL 


f~ a j en A 
a NE cee ae Peer aa eens Pera EC gat 


ae _— 


~~. 


co SSVSHONIY 2 
4SV¥3Y030 


—_— 


ae NMOG on 
(WOO HOLVH3DIN43Y) | 99V | 


oN SNg0 


{ 3NOYOL » | 
Tin rl ey wi 
7 NS 


¢ 
an (WOOY “o9) 95y a 3SV3HONI| _Ss3auis 
ON peel en PRN 3SV3H030 379NV W13H 
p \ eT) WEN gover Sere 


2 SA 
. OV4UNS 3AVM ite 
Sos a A a tad Ga dua putas eouoit list aes ada. TUE a ee ar La 


(1) H3QYOD-ISIA NO GHOO3Y 


78 


Table 3.1 summarizes the measurements of the responses and shows that these 
thirteen representative runs cover a good variety of sea conditions, wave heights, and 
directions. This is reflected in the variation of normalized correlations and spectra, for 
example in the rolling, pitching, stress, and relative wave heights in Fig. 3.14 and 
ign sels: 


If we assume a variety of environmental conditions, it is possible that various sea 
conditions might appear ideally by the same chance, and the average of all normalized 
wave spectra might give us a nearly white spectrum, or the spectrum of white noise. Then 
the averaged response might show the characteristics of the response to the white noise, 
i.e., the characteristics of the response functions themselves, as was discussed by 
Yamanouchi, et al.*° The average normalized correlation diagram or correlogram and 
averaged spectra are shown in Fig. 3.16. 


These averaged rolled and pitch spectra are not the ideal ones, but quite reasonably 
show the smooth peak at its natural frequency. Moreover, the correlation functions of 
roll, pitch and stress show the beautiful forms of damped oscillation. The averaged spec- 
trum of stress has a smooth peak at twice the frequency of the roll natural frequency. The 
natural frequency of the stress response should be in a higher frequency range, but must 
have been cut off by the filters and should not appear in the averaged spectra. The peak 
that does appear at the double frequency of the roll natural frequency might indicate that 
the transverse stress induced at the web frame is quadratic nonlinear to the rolling motion. 


To check this possibility, an artificial process of rolling square was made as shown 
in Fig. 3.17. This naturally has a biased mean as shown in its variational form in the time 
series and also in the spectrum in Fig. 3.18, where the power value near w = 0 is large. In 
the next step, a digital high pass filter, shown by Eq. 2.132, was applied to this roll 


1 
squared process as X,(f) = ry {xa =x) 4}. The filtered roll squared process appears at the 


bottom of Fig. 3.17 and its spectrum in Fig. 3.18. In Fig. 3.18, we find the shape of the 
spectrum of the filtered roll squared process is similar to that of the averaged spectrum 
of the stress, which validates the assumption that the stress is a quadratic and nonlinear 
response to rolling motion. 


The lower two graphs in Fig. 3.19 show the results of single input-output spectrum 
analysis, with the stress as output and relative wave height or rolling as a single input. 


Coherencies 7” values are so low that the stress cannot be the output of only the relative 


wave height nor only of rolling. The top of Fig. 3.19 shows on the contrary that relative 
wave height is fairly well explained just by rolling; the coherency, especially in the 
frequency range in which rolling has reasonable power, has a value pretty near 1. 


Since we found that the stress was quite possibly quadratic and nonlinear to the 
rolling, the next step was to find the response of stress to the single roll squared process, 
(roll)?. To eliminate the effect of large bias on (roll), the response of stress to the single 
(roll)? filtered process was also obtained. The results are shown in Fig. 3.20, and the 
coherency is again rather low. 


To check the coexistence of many other inputs, multiple input analysis was intro- 
duced. Stress was considered as the output of rolling, pitching, relative wave height, and 
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Table 3.1. Summary of seaways and particulars of measurement 
and analysis for test runs. 


(From Yamanouchi.?3) 
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Fig. 3.14. Example of auto correlations (normalized) and the spectra (run 207). 
(From Yamanouchi.*%) 
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Fig. 3.15. Example of auto correlations (normalized) and the spectra (run 214). 
(From Yamanouchi.°%) 
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Fig. 3.16. Averaged correlograms and spectra (normalized). 
(From Yamanouchi.?3) 
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Fig. 3.17. Roll, (roll), 1/2{(roll),,2 — (roll),,17} processes. 
(From Yamanouchi.?%) 
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T.NO. 207 


Sp2p_2FILTERED 


Fig. 3.18. Spectra of roll, (roll), (roll)*filtered. 
(From Yamanouchi.??) 


heaving. Heaving itself was not measured, but its effect was assumed to be included in 
the vertical acceleration of a representative point and also to be combined in pitching, 
rolling, and relative wave height. Examples of four—input analysis are shown in Figs. 3.21 
and 3.22, in which stress as the output is given on channel 5 and the inputs of rolling on 
channel 1, pitching on channel 2, relative wave height on channel 3, and (roll)? filtered on 
channel 4. Figure 3.21 shows five auto correlations of output and input Rss, Ry;, R22, R33, 
Rag; ten cross correlations of combinations of the one output with four inputs Rs), Rs2, 
R53, R54; R21; Rai, R42, R43; R31, R32; and five auto spectra of stress Sss, rolling $1, pitch- 
ing S22, relative wave height S33, and roll squared. Figure 3.22 shows conditional (partial) 


gains and phase relations as Hop pwr2, Hsp_pwr2, Hsw_ppp2, and Hsp2_ppy in the form 


of absolute values and arguments. The notation follows the convention already explained 
in Section 2.6.5 on partial coherencies. For example, 


Ssp_RWRW) 
H 2 (CO) ere 
SP-RWR2®) ame) 
shows the frequency response of stress to R: rolling, W: relative wave height, and R?: 
squared roll filtered processes under the condition that the effect of pitch has been 
masked. In the same way, multiple and partial cross spectral analysis was tried with stress 
as one output (4), and roll R (1), pitch P (2), and (roll)? filtered R? (3) as three inputs. In 
another one output/three input cases, pitching P was replaced by vertical acceleration A. 
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Fig. 3.19. Coherencies for single input—output relations. 
(From Yamanouchi?) 


The results of spectrum analysis of the first three—input case are shown in Fig. 3.23 and 
Fig. 3.24. 


Auto correlations R44, R33, R22, Ry, and all combinations of cross correlation R4), 
R42, R43, R31, R32, Roi and auto spectra S44, S11, S33 are shown in Fig. 3.23. Partial gains 


and phase relations Hsp _pr2, Hsp_pp2, and Hsp2_pp are shown in Fig. 3.24. The effects 


of the increase in probable inputs are more clearly shown in the multiple coherencies, and 
the effects of several inputs when the effect of a certain input was masked are shown on 
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Fig. 3.20. Coherency of squared rolling process and filtered squared 
rolling process to the stress. 
(From Yamanouchi.??) 


the partial coherencies. The results for two—input, three—input and four—input cases are 
compared in Fig. 3.25. 
This figure indicates that the multiple coherency shows higher levels as more inputs 
. ome . . 2 
are taken into account. Moreover, examining the partial coherencies, y SR-PWR?? 


Ysp?-_rpw and others, at the bottom figure of Fig. 3.25, we find that the contributions of 


rolling and of (roll)?-filters are large compared with the other inputs. This finding sup- 
ports the assumption of quadratic nonlinearity of transverse stress to the rolling motion. 


Figure 3.26 shows the multiple and partial coherencies for the two—inputs and 
three—inputs cases, and here again we find the coherency improves with increasing num- 
ber of inputs, including the (roll)—filtered process. Also the contributions of rolling and 
(roll)*—-filtered are large, compared with the contribution of pitching. The multiple coher- 
ency is very close to 1 around the frequency range in which the output, here the stress, 
has important response to input. 
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Fig. 3.24. Example of three—inputs analysis; amplitude gain and phase shift. 
(From Yamanouchi.??) 


To show more clearly the usefulness of multi input analysis, apart from the nonlin- 
earity of responses, another example used the vertical acceleration as a linear output and 
rolling, pitching, and relative wave height as inputs. The multiple and partial coherencies 
are given in Fig. 3.27, in which the multiple coherency is almost 1 at the important range 
of frequencies and the contribution of rolling is large for this vertical acceleration, as we 
expected. 


Figure 3.28 is another example of two—input and three—input analysis for stress 
which does not take into account the nonlinearity of rolling. Comparison with Figs. 3.25 
and 3.26 at the important range of frequencies for stress shows lower coherencies when 
the nonlinear response of rolling is not taken into account. 


Later, the author found that taking the input process squared as an input is very rea- 
sonable in discussing the quadratic response character of the output as shown in Section 
11.4, in Part II. 
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Fig. 3.25. Examples of multiple and partial coherencies — (I); stress—roll, pitch, relative 
wave height, and (roll)?; comparison of two-, three—, and four—inputs cases. 


(From Yamanouchi.?%) 
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Fig. 3.26. Examples of multiple and partial coherencies — (II); stress—roll, 
pitch, and (roll)?; comparison of two— and three—inputs cases. 
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Fig. 3.28. Example of multiple and partial coherencies — (IV); stress—roll, vertical 
acceleration and relative wave height; comparison of two— and three— 
inputs cases, when the (roll)? is not taken into consideration. 


(From Yamanouchi.?3) 
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CHAPTER 4 


CONCLUSION FOR PART I 


In Part I, the nonparametric procedure for analyzing the irregular time series was 
discussed. At the beginning of Chapter 2 the basic procedure for obtaining the spectrum 
through periodograms was summarized and then a few considerations for improving this 
method were introduced in Chapter 3. The conclusions derived from the discussion are as 
follows: 


1. The spectral analysis technique with the use of periodograms is well established 
and affords us a powerful approach for analyzing irregular phenomena. 


2. In sample computations of the correlation function as well as of the spectra, 
however, care is necessary in sampling the data. The sampling time interval, the length 
of record, maximum length of lag, and proper windows must be chosen to get consistent 
estimates and avoid aliasing, blurring by the windows, or loss of reliability. 


3. Spectral—lag window pairs have been proposed by many scholars, and were 
shown that we must be careful of the effect of windows on the reliability, variability, and 
resolution of the results. 


4. Computation of the spectrum through periodograms by use of the Finite Fourier 
Transform is the same as that computed through the correlation function. 


5. The Fast Fourier Transform (F.F.T.) method is a convenient algorithm for reduc- 
ing the number of operations in the computation, but the considerations of the windows 
are also necessary and important in applying this method. 


6. The correlation functions can also be conveniently calculated by the F.F.T. 
method if proper precautions are taken. 


7. In connection with the choice of windows, the use of filters before applying the 
window is worth considering. 


8. Not only the spectrum functions but also the correlation functions (correlo- 
grams) should be investigated carefully in estimating the character of the process. 


9. In applying the spectral window in cross spectral analysis or in response analy- 
sis, the shift of the origin of the cross correlation (shift of the output record) should be 
considered to minimize the leakage of power through the use of spectral windows in 
computing cross spectrum. 


10. Cross spectral analysis is essential in the analysis of the response process of a 
dynamic system to get full information on the frequency response functions of the sys- 
tem. Cross spectral analysis is effective in reducing the effect of noise that contaminates 
the output. 

11. The coherency function is a good index to the extent to which the response can 
be expressed by linear relations. To make the coherency function useful the computations 
of all spectra must be done properly according to the preceding items 4~10 in getting the 
coherencies. 

12. The impulse response function may be obtained from the cross and auto corre- 
lations of output and input, without the trouble of choosing a window. This procedure 
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was found later to be closely related to the AR—model fitting technique discussed in Sec- 
tion 5.5. 

13. In the analysis of seakeeping data, multiple input analysis is helpful. Partial co- 
herencies and the multiple coherency give a good clue to the extent of contributions from 
each input. 

14. In connection with consideration of the filtration of the stochastic process, the 
model fitting techniques that try to express the stochastic process by a finite number of 
parameters are found to look promising to supplement the nonparametric method. The 
model fitting technique or parametric method will be reviewed in detail in Part I. 
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PART I 


MODEL FITTING TECHNIQUES 
(PARAMETRIC SPECTRAL ANALYSIS) 


CHAPTER 5 


DISCRETE MODEL, MODEL FITTING, 
CORRELATION AND SPECTRUM FUNCTIONS 


5.1. INTRODUCTION 


In Part I, the procedures for estimating the spectrum nonparametnically through 
autovariance from a sample observation were summarized and discussed. We noted that 
the statistical consideration of each step of the computations are important in getting 
reliable results, and the author suggested several ways to improve reliability. In Part I, 
another approach to getting reliable results from a single or a short observation of a 
process will be discussed. The method introduced here is called a parametric approach, 
because some type of model is fitted to the sample observation, and then the parameters 
of the model are estimated statistically. In looking for the models, we can use the knowl- 
edge that we already have about the process, such as the degrees of freedom, and the 
physical characteristics of the equations of motion that govern the behavior of the system 
or about the inputs. A few methods which use the parametric approach, such as the 
maximum likelihood method (MLM) and the maximum entropy method (MEM) are 
essentially the same with the one mentioned here, and will be explained later. 


In this analysis, the criteria for deciding the fitness of the model are very important. 
The method introduced here, called the “‘“MAIC Method,” was introduced by Dr. H. 
Akaike of Japan and provides a powerful guide in finding the properly fitted model, prop- 
er from a statistical point of view. In this analysis, the time domain expression of this 
process, the time histories themselves, and the correlation of the processes play a big role 
as was pointed out in Sections 1.3 and 2.5.8 and in the conclusion to Part I. 


This parametric approach is not yet well known in the field of naval architecture, 
although a few books by Priestley,2> Pandit and Wu,*° Box and Jenkins,?’ and others*® 
deal with it in part or in depth. For this reason Sections 5.2.1 to 5.2.5 give several statisti- 
cal models of stationary time series in some detail with simulated examples by this 
author. Then criteria for choosing the model, estimating the parameters, and deriving 
the spectrum will be given at the end of Chapter 5. 


Chapter 6 discusses the application of this method to a two—variate process, an 
input/output system, and the usefulness of this method for the analysis of a response 
system with feedback is shown. 


In Chapter 7, examples of the application of this method to the analysis of seakeep- 
ing data and a comparison with analysis by the nonparametric method are shown. 
Finally, Chapter 8 provides conclusions and summaries for Part II. 


OF, 


5.2. DISCRETE PARAMETRIC MODELS 

There are many statistical models for expressing time series. Among them the auto- 
regressive (AR) model, the moving average (MA) model, and the mixed autoregressive 
moving average (ARMA) model are the most representative linear models that we en- 
counter in the analysis of irregular records of observations. In Part I, only the linear 
models are introduced. Some types of nonlinear models will be referred to later in Part 
Ill. The order of the AR, MA, and ARMA models indicates the degree of simplicity or 
complexity of the models. 

Here in Part II, the discrete time process sampled from the continuous time process, 
with an interval Ar is expressed by {X;} and its realization by X;. Except as otherwise in- 
dicated, Ar is taken as 1. If Ar = 1, each function is easily transformed to proper form, as 
was already mentioned in Section 2.4.3. The explanation of the character of the elemen- 
tary models is based largely on the work of Priestly”? and Pandit and Wu.*° 


5.2.1 Pure Random Process 


{X;,} is called a pure random process if it is the sequence of uncorrelated random 
variables that are stationary up to order 2, and is written 


X,= €;. (5.1) 
The mean is 
E|X) =n, (5.2) 
the variance is 

E\&,-n)"| = 07 = 02 (5.3) 

and the covariance function is 

0 rz0 
= 2 (5.4) 
R(r) cov.{X,, Ge 32 cnc se 


R(r) is a function of r only and is normalized by a? as 


o(r) dk 0 r= 0 (5.5) 
ip 
Its spectrum is then 


fee) 


1 z 
s(w) Se > Rien =< = const. (for-z7 =w2z7). (5.6) 


r=0 
In this case 
1. When this process is also Gaussian, then X; is not only uncorrelated, but also 
X;, X;1,° + *X;-,, are independent of each other. 


2. The spectrum s(w) is flat for the interval of frequency w for —z toz, and is 
referred to as white noise. 
3. The process is also referred to as a memoryless process. 
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Bagi as © 


Fig. 5.1. Theoretical autocovariance of a pure random process. 


Fig. 5.2. Theoretical spectrum of a pure random process. 


52.1.1 Example of a Pure Random Process. Figure 5.3 shows an example of a 
pure random Gaussian process generated fort = 1 — 600, with average 0 and variance 
1.0. At the bottom, a part (t = 100 — 250) of the record is shown expanded on the time 
axis. Its readings are listed in Appendix Al as Table Al.1; pp. 251, 252, and 253, for 
reference. This process was generated by the method of “Random number generation and 
testing,” generally popular as “‘multiplication type residual method.” 


It was found that by this method it is hard to get a really white process with precise- 
ly designed variance of 02 = 1.0 from this short record (N = 600), and the sample 
variance appeared as G2 = 1.046490. Figure 5.4a shows the theoretical autocorrelation 
coefficient 9(0) = R(0)/R(0) = 1, o(r) = R(r)/R(O) = 0, r = 0 by Eq. 5.5. Figure 5.4b 
shows the estimated Q(r) = R(r) /R(0) from the generated process. From the model fitting 
technique and order determination that will be mentioned in Section 5.5, the order ap- 
peared to be 0, and AR(0) appeared to be the most appropriate model to fit this process. 
The estimated autocorrelation of this model fitted by Eq. 5.4 is just the same with the 
theoretical o(r) shown as Fig. 5.4a because the difference is only ing, values, that its 


drawing was omitted. Figure 5.5a shows the theoretical spectrum function of this process 
designed by Eq. 5.6, that is, the white noise. Figure 5.5b shows the estimated spectrum of 
the model fitted by Eq. 5.6. This is also the white noise and looks very similar to the 
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Fig. 5.3. Simulated pure random process AR(0) 
X=€1,  €: N(O, 1]. 


theoretical spectrum except that the variance was a little larger, G2 = 1.046490, as 
mentioned above. 

In Fig. 5.5c, for comparison, the estimated spectrum, calculated by the nonparamet- 
ric or correlation method, and the Fourier transform of the calculated Rr) in Fig. 5.4b are 
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0.00 = 


-0.50 = 


-1.00 


I ' U i ! i ' 
0.00 7.50 15.00 22.50 30.00 37.50 45.00 52.50 60.00 


= 


Fig. 5.4a. Theoretical AR(0). 


1.00 


| ' | 
0.00 7.50 15.00 22.50 30.00 37.50 45.00 52.50 60.00 
5 


Fig. 5.4b. Estimated. 
Fig. 5.4. Autocorrelation coefficient for pure random process AR(0) 
X= €+, Er: N[O, 1]. 


shown. The maximum lag number M = 60 and the so—called Hanning Type spectral win- 
dow was used to calculate this spectrum. It looks very different from the white spectrum 
and has many sizable peaks and valleys. In the figures, the 90% level of confidence inter- 


val of this estimate, based on the y” —distribution of the equivalent degree of freedom 
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(here approximately equal to 27), as is explained in Section 2.5.5 is shown as vertical 
lines for reference. 


2.50 5 


2.00 4 oe = 1 


Fig. 5.5a. Theoretical AR(0). 


2.50 


6. = 1.046490 


0.50 


0.00 


Fig. 5.5b. Estimated by model fitting as optimum AR(0). 


Fig. 5.5. Spectrum for pure random process AR(0) 
X= €1, Ee: N[O, 1]. 


This difference tells us that the process might not actually be white in this short 
period of 600 and also that it is hard to get the ‘real’ character of the spectrum by the 
nonparametric method from this short record. It is interesting to note, however, that by 
the model fitting method, even from this short record, the fluctuations disappeared and 
the spectrum appeared to be white. 
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0.50 4 


(ne 
0.00 0.06 0.12 0.19 0.25 0.31 0.37 0.44 0.50 


Fig. 5.5c. Estimated by nonparametric method. 
Fig. 5.5. (Continued) 


5.2.2 Autoregressive Process of the First Order, AR(1). 
When X, follows the relation 
X,=aX,i+€;, (5.7) 


where a is a constant and {e,| is a stationary pure random process, 


2 ets 
then GEG ee ees SO Gig.) = GB) 
0 otherwise. 


{X,} is called an autoregressive process of the first order AR(1). Equation 5.7 means that 
1. X, has a linear regression on X;_) as in Fig. 5.6 
2. €, plays the role of error in the above relations of regression 


3. X, is dependent on one step back value of the same process X;_1. 
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> X< 


0 > Xo 


Fig. 5.6. X, vs. X,,. 


Besides, when {X;,} 1s Gaussian, this process is called a ‘“‘Linear Markov Process” as 
will be explained in Part III. 


4 


Fig. 5.7. €,; VS. €:,. 
5.22.1 Green’s Function of AR(1). 


XxX; = €;+ aX} 


= €,+ a(E;_] + aX;_2) 


=: 


=€;+a €,1+-a" €pot-: +a Hey +aXo} 
t-1 

= y a’ €,,+a' Xo, (5.9) 
r=0 


If initially Xo=0, or X,=0 when r < 0, 
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then XOS y BE (5.10) 


or, considering the stationarity of X, and assuming that X_, exist, 
t 
xXGS » a'~“e,+a*NX 4. If X_n is 0, when N tends to 
u=~(N-1) 


t oe) 
GS SES ae (5.11) 
j=0 


u=-@ 


which is the same form as Eq. 5.10. Setting 


Gj=ai (5.12) 


gives 


[--) t 
X;= Sy Gj erj= Sy, G,.j €j- 638) 
j=0 


F=—0O 
G; is called Green’s function for AR(1). 

Green’s function G; shows the weight given in the present response of X; to the dis- 
turbance €,_;, which entered the system j time units back. It also indicates how well the 
system remembers the disturbance €,; or how slowly or quickly the dynamic response of 
the system to any particular €,_; decays. Equation 5.13 is called Wold’s decomposition 
and gives the decomposition of X; into an infinite number of orthogonal variables G; €,_; . 
Equation 5.13 also implies that the AR(1) process can be inverted into an infinite order 
moving average process MA (~) as 

Xe = Ge; Gi €ni+ G6 fot G ee Ee Galsy) 


5.22.2 Solution of the Difference Equation. Equation 5.7 is a difference equation 
and can be written 


X,— Xx} =€;. G79) 


The general solution of the difference equation is the sum of the solution of its 
homogeneous equation plus a particular solution of Eq. 5.7’. The homogeneous 
equation of Eq. 5.7’ is 

X;—aX,; = 0, (5.14) 
its solution being 
Xo—An (5.15) 


where yw is the root of its characteristic equation, equated to zero. 
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fZ) =Z-a=0, (5.16) 


namely 
M=a. (5.17) 


A is an arbitrary constant and is determined by the initial conditions. From Eqs. 
5.15 and 5.17, 


X,= Aa’. (5.18) 


One particular solution of Eq. 5.7 is obtained as follows using the backward shifting 
operator B. Equation 5.7 is 


(1 —aB) X;=€;, (5.19) 
X,=(1-aBye,= 5 (@/ Bye, = > ai €,;. (5.20) 
j=0 j=0 


This equation is the same as Eq. 5.10 or 5.13 expressed by Green’s function. Therefore, 
the general solution of Eq. 5.7 is 


[e.e} 
X,=Aa'+ >. a! €1.; (5.21) 
j=0 
The first term that is the solution of the homogenous equation is the free oscillation of 


this system. When lal < 1, this free oscillation decays, and only the second term remains 
as a Stationary oscillation as in Eq. 5.20. Equation 5.19 is, more generally, 


a(B)X, = €;, (5.22) 
X,=a7 (Bye, (5.23) 
where 
a(Z) = 1-aZ. (5.24) 


In order to have the free oscillation damp out, lal < 1 is necessary and, with the character- 
istic Eq. 5.16 equated to zero, {Z) = 0 must have its root inside the unit circle. Then from 
the equation 

a(Z) = 0, (5.25) 
or 1—aZ =0, its root is the reverse of the root of the equation Z—a = 0, or fiZ) = 0. We 
find a(Z)=0 must have its root outside the unit circle. 


522.3 Inverse Function of AR(1). Green’s function can be considered as an indi- 
cation of how X; can be expressed by the MA (©) process, because X; is expressed as the 


summation of infinite €, at preceding time points. 
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j=0 


= Go€, + Gi€,1+ Gre, 2+--- +Gj Ewjt+--- 
= (GoB° + G\B + G>B* + - ° HG; Bia --) €, 
j=0 
Usually Gp = 1. As the inverse relation to this Green’s function, if 
€,= > (-JjB!) X; 
j=0 
= Clg iB =p Bae : -—1,Bi+- . )X, 


= =NG—Ih G5 Ih G0 > oI Nep—o oo, (G20) 
functions Jp, /;,- - -J ; are called inverse functions, and usually —/p = 1. 
Therefore 
€,=X,-1,X,.-1 —1nX,2-- 9 9) FO. 0 Ones (5.27) 


This inverse function shows how X; can be expressed by AR(©). Therefore the 
inverse function of a pure autoregressive process AR(n) actually has no meaning. For 
example, for AR(1) as in Eq. 5.7; 


Ip =- 1, I, =+a, I; =Oforj = 2. (5.28) 


5.2.2.4 Stationality of AR(1). From Eq. 5.10 


t-1 
X;= lim y, We, = €,+ GE, 1+ a°€,2+- : EE 
FS A) 
Then if 
Ele,] =e for all t, (5.29) 
and as €; represents white noise, 
2 
0. t=u 
Ele;-eoll= sees ; 
[Er €x] 0, bye (5.30) 


we can get 
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Me a1 
E[X,] = lim Me [I1ta+a’+: +a} = lim l—a » (5.31) 
oe eo) 1 © 
Me t a=1 
t-1 : t-1 
cov. [X,;-X,4,] = lim ») ae, _; Sy GE rs pj 
be i=0 j=0 
= lim ota’ +a"? + rar e2e-o} : 
to 
1 2t 
oa’ 2 ail 
Therefore R(r) = lim ie : (5.32) 
1 © 
o2t, a=1 
If ue = 0, 
1—a~ 
oz => @ #1 
var. [XJ=R0)=lim4 1-4 (5.33) 
t— © 
o2t, a=1 


From these results: 
[a.] When. = 0 


E[X,] is, from Eq. 5.31, a function of time and therefore {X;} 1s not stationary, even 
to order 1. 


(1.) When lal < 1, and when t tends to a large value 


E[X,;] = = = const. by ¢. (5.34) 
Therefore {X,} is asymptotically stationary to order 1. 
(2.) When a= 1 
E[X;] =Met (5.35) 


E|X] increases by time ¢. Therefore, {x,| is not stationary, even up to order 1. This is the 
case of Random Walk. 


[ b.] When, =0 
E[X,] = 0. 


cov. [X;- Xr] = Rr) = 2a" aE 
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(1.) When lal < 1, i.e., when {Z) = Z—a = 0 has its root inside the unit circle or 
when @(Z) = 1—aZ = 0 has its root outside the unit circle, from Eq. 5.32, even E{X;} = 0 
holds, the cov. [X,;, X, 


up to order 2. However, when t—> & , 


is a function of time. Accordingly, {X;} is not stationary even 


+7] 


2 
0. 
cov. [X,-X,,,] =RW)= i 5 a’ const. by t, (5.36) 
-—a 
a2 
ar. [X;] = E[X, X,] = R(O) = const. by 1. (5.37) 
-—a 


Therefore, {X;} is asymptotically stationary up to order 2. The form of R(r) will be 
different by the sign of a, although it gradually decays as increases as in Fig. 5.8. 


Sale 


(a)O0<a<1 (b) -1<a<0 


Fig. 5.8. A(A of AR(1). 


(2.) When lal > 1, i.e., when f(Z) = Z—a has its root outside the unit circle, or when 
a(Z) = 1 —aZ has its root inside the unit circle, 


2 
cov. [X; - Xu,] = R(r) = = i a’(a—1) (5.38) 
oO 2 
var. [X= Ai GEN) (5.39) 
a — 


In this case, E[X] = 0, but not only does R(r) not converge to a small value as r increases, 
but both R(r) and R(O) change by the time ¢ and are not stationary. As time passes, these 
values continue to increase. Accordingly, this process is not stationary up to order 2, even 
asymptotically. 


5.2.2.5 Autocovariance and Spectrum of AR(1). The general solution of R(r) can 
also be obtained more simply as the solution of a first order homogenous difference equa- 
tion, because in this case, from Eq. 5.7, 


X, = aX;1+€1, (3.7) 
assuming 4“ « = 0 and E[X,] = 0. Multiplying both sides of Eq. 5.7 by X,_, and taking the 
expected values gives 
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R(r) =a R(r-1) when r ~0 (5.40) 


R(O) = a*R(O) +02 when r = 0, (5.41) 
because 


ages 
E[X;- €;]= hii (5.42) 


Ele?]=o2 i=], 


Equation 5.40 is the same type of homogeneous difference equation as that of the original 
process {X,} shown in Eq. 5.14. 


R(r)—aR(r—1)=0. (5.43) 
Using the backward shift operator B gives 

(1-aB)R(r) = 0. (5.44) 
Thus R(r) = (1—-aB)'R(r) = 0 (5.45) 
and R(r) = Aw’. (5.46) 
Here x is the root of f(Z) = Z—a =0 when n=a 
and thus R(r) = Aa’. (5.47) 


In order to be asymptotically stationary, R(r) — 0 as(r) — © when lal < 1 and f(Z) =0 
must have its roots inside the unit circle. 


From Eq. 5.41 
2 ae 
RO)=0;=Az= >: (5.48) 
l-a 
Therefore, the general solution is, from Eq. 5.43, 
o2 
RN) => a" = R(0)a’ = 020". (5.49) 
—a 


This equation is the same as Eq. 5.36. 
522.6 s(w) of AR(1). Here we assume u, = 0, lal <1. 


Then the spectrum s(w) is obtained as the Fourier transform of R(r) 


sw) == SY ROea 


r=-—-@ 


u = R(0) +2 Y. Re) cosrw 


r=1 
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Substituting the values of R(0) and R(r) in Eqs. 5.48 and 5.49 gives 


co 
o2 1+2 S a’ cosrw 


r=1 


GE i ; 
or 1+2 SS Re {are | 


r=1 


o- 9 . P 
= = F + 2'Ke {aei# + (ae)? + (ae)? +- - 1] 


2 +m 
EEF 1.9) Cad 
—aet 
1 +iasi 
se cele Se acos@ ase 
270 (1-—a cos w)-—i asin w 
Ge 14 2 A L08 WU =a cos) ~ a* sin’ w _& ar 
On (1—acosw)*+a*sinw 29 1—2acosw+a-’ 
here Re {-} indicates to take the real part of a function {-}. 
; az 
Inserting Eq. 5.37, that is 02 = = gives 
oz 1 
SO) = es 
22 (1—2acosw +a*) 
2 2 
O, 0. 
= £ = = (5.50) 
on | 1-ae|2 a | ave)? 
Therefore, 
2 2 2 
Ox 1- Ox 1+ 
5(0) = 2 a : _Oxita 
2x (1-a) 2x 1-a 
2 
Get 3 (5.51) 
27 (1 —a) 
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laa oe @ il 
a2nx 2n(1+a) 


S(l) = S(— 7) = #52) 


Its shape will then be as shown in Fig. 5.9. 


S(w )/S(0) 


Fig. 5.9. s(@) of AR(1). 


5.2.2.7 Estimation of a and oz. When we are given a set of data, X;- - - Xy, with 
numbers N, and the AR(1) model is to be fitted (the determination of order will be dis- 
cussed later in Section 5.5), we can easily estimate the values of a anda? by the 
minimum least squares method as 


N 
»S X; X14 
ga (5.53) 
OX 
t=2 
and the minimized residual sum is 
N 
Ue 7 wat (X,-4 Xia). (5.54) 
The same results can also be derived ae es 5.40 and Eq. 5.41 as 
pO) 
~ RO) (5.55) 
and 62=(1-—d7)R(0). (5.56) 


Equations 5.53 and 5.55 have the same content as if the sample correlation were replaced 
with R(0) and R(1). Obtaining a by the linear minimum least squares method from Eq. 
5.53 is actually done by Eq. 5.55. Then, the variance a? ofe, is estimated by Eq. 5.56. 


5.2.2.8 Example of AR(1). Figure 5.10 is an example of the AR(1) process simu- 
lated by an AR(1) model X,—0.5 X,_) =€,, where a = —0.5 in Eq. 5.7. In order to look 
at the pattern of variations, at the bottom of this figure, a part (t = 100 to 250) of this pro- 
cess is shown expanded on the time axis. Its readings are listed in Appendix Al as Table 
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Fig. 5.10. Simulated AR(1) process 
X,-0.5 Xey =€Er, Er: N[O, 1]. 


A1.2; pp. 251, 252, and 253. Heree€, is a pure random Gaussian process N[0, 1], and the 
same process, which was generated as a pure random process AR(0) in Fig. 5.3, was 
used. Figure 5.11a shows the theoretical autocorrelation coefficient @(r) = R(r)/R(0), 


from Eqs. 5.36 and 5.37 using the designed value of 0? = 1 anda = —0.5. Figure 5.11b is 


the estimated autocorrelation function 6(r) = Rr) /R(0) from the simulated process in Fig. 
5.10. An AR model was fitted to this process, and the order actually obtained was 1 by 
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-0.505 
Oa a er aT ee TU 
0.00 7.50 15.00 22.50 30.00 37.50 45.00 52.50 60.00 
7T 
Fig. 5.11a. Theoretical. 
1.00 


> ' i} 
0.00 7.50 15.00 22.50 30.00 37.50 45.00 52.50 60.00 


Fig. 5.11b. Estimated. 


Fig. 5.11. Autocorrelation coefficient for AR(1) process 
X,-0.5 Xe =€;, Er: N[O, 1]. 
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the method of order determination that will be mentioned later in Section 5.5. Now with 
the order as 1, a anda? were calculated from Eqs. 5.55 and 5.56, as @ = — 0.50933 and 


$2 = 1.04646, which are close to the a = — 0.5 anda? = 1.0 used to generate the process. 
We also estimated the correlation coefficient 6(r) = R(r) /R(0) from Eqs. 5.36 and 5.37, 
using these @ and Ge . These values are so close to the theoretical R(r), or just the same in 
o(r) = R(r)/R(O) in Fig. 5.11a that their drawing was omitted. Figure 5.12a is the theoret- 
ical spectrum from Eq. 5.50, using the design values of a = -0.5 anda? = 1. 


4.374 


0.06 012 019 O25 O31 40.37 0.44 ~°& 0.50 
7 @ 


Fig. 5.12a. Theoretical AR(1). 
Fig. 5.12. Spectrum for AR(1) process X;—0.5 X,,=€;, €+: N[0, 1]. 


The estimated spectrum s(w), with d = — 0.50933 and? = 1.04646 obtained by 
model fitting and also by Eq. 5.50, is given as b in Fig. 5.12. For comparison, as c in Fig. 
5.12, the spectrum 5(w) was also estimated by the nonparametric method as the Fourier 


transform of R(r) in Fig. 5.11b, using the Hanning window and maximum lag M = 60. It 


is interesting that the spectrum estimated by model fitting is very similar to the theoretical 
spectrum, Fig. 5.12a, of this generated process. On the other hand, the spectrum esti- 
mated by the nonparametric method Fig. 5.12c is more wavy than the theoretical one, 
although the fundamental shape is similar. The wavy fluctuations were found to come 


mostly from the wavy fluctuation of the input white noise {€,} that is shown in Fig. 5.5c, 


as the incomplete whiteness of the pure random process {€,} generated. In the figures, 
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c& =—0.50933 
a. = 1.04646 


0.00 
0.00 0.06 012 0.19 025 031 0.37 044 0.50 
-@ 
Fig. 5.12c. Estimated by nonparametric method. 


Fig. 5.12. (Continued) 
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the 90% level of confidence interval of this estimate, based on the y*—distribution of the 


equivalent degree of freedom (here approximately equal to 27) as is explained in Section 
2.5.5 is shown as vertical lines for reference. 


5.2.3 Autoregressive Process of the Second Order, AR(2) 


As was shown in the last section for the AR(1) process, the residual part €, of X;, 
obtained by subtracting the linearly dependent part aX,_; from X 1, 


€,= X;—aX;1 Gia) 
was a purely random process uncorrelated with€ 1, €;2: - - aS in Fig. 5.13a and uncor- 


related with X;_2, X;3- - - aS in Fig. 5.13b. 


0 > e., 


Fig. 5.13a. ©, vs. €,_, for AR(1). 


X2 


Fig. 5.13b. ©, vs. X,-> for AR(1). 


Fig. 5.13. Characters of €, for AR(1). 


Go = 0 


El€,- €++) = Ele,-X;2] = 0. 


If this relation does not exist and the residual €;, that is, 


€;/ = X,-a'X,-4, (5.57) 
was linearly correlated with X,_2 as in Fig. 5.14, then 
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€;/ = a, X,2+€,. (5.58) 


The residual part €, =€, —a3 X;_2 is now a purely random process. Then inserting 
Eq. 5.57 into Eq. 5.58 gives X;—a’X;_) — a3 X;-2 = €;. 


X12 
=> 


Fig. 5.14. €, vs. X,_. for AR(2). 


Here changing —a' = a), —a3 > ap, 
Xp t+ QyX7_1 + Q2X72 = €;. (5.59) 
The process, which satisfies Eq. 5.59, is called the autoregressive process of the sec- 
ond order. Here, a; and a2 are constant, {€,} is a stationary pure random process, and we 
assume E[e,] = 0. 
5.23.1 Solution of AR(2), Green’s Function for AR(2). Starting from Eq. 5.59 
X,+ @,X;-1 + A2X1-2 = € 1, 
we can solve X; as the general solution of a second order difference equation, i.e., a 


general solution equals a complementary function plus a particular solution. A comple- 
mentary function is the solution of the homogeneous equation 


XxX; + Q4X1-1 + a2X 1-2 = 0. (5.60) 
The solution of this homogeneous equation is in the shape of 
Ay yi +A2 1. (5.61) 


(41,42, being the roots of the characterisitc equation f(Z) = 0. The particular solution can 
be obtained, using the backward shifting operator B, as 


(1 +a,B + a>B?)X, = €;. (5.62) 
Setting 
Q(Z) =1+a\Z+a7Z’, (5.63) 
a(B)X,= €;. (5.64) 
If the quadrant fD=Z +a\Z+a,Z =0 (5.65) 


has two roots 41,2, 
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| 1— (U4 + 2)B + uyeB?}X, =€,, 


Hi th2=—-a, 
M2 = 22, 


(1 —4)B)(1 —M2B)X, = €,, 
then 


Cal €; fu 1 
~ Q(B) (1+a,;B+a2B*) (1 — w,B)(1—2B) 


Pett eee ie Ee og eee 
= ae t 
Mi-H2 | 1-418) (1—-p2B) 


X; is a particular solution. Therefore, if the expression of Green’s function G; is 


used, 
X;= is Gj ery 
j=0 
J+1 jri 
Thus Ce ee) 


TT Ra pS pain’ 


= 21 Mit 82 M3, 
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t 


(5.66) 


(5.67) 


(5.66’) 


(5.68) 


(5.69) 


(5.70) 


This solution X, can also be written as 


foe} 
KS SG 9 Cae Cie, pt Gasp a? 2 (5.71) 
j=0 

Equation 5.71 shows the AR(2) process is expressed by MA( © ), as was the case for 
AR(1). 

5.2.3.2 Inverse Function for AR(2). To say the inverse function for the autoregres- 
sive process has little meaning was pointed out for AR(1). Formally, however, for AR(2), 
from 


X,+ a)Xj-1 + 22X12 = Er, 


(1 +.a;B + apB)X, = CW, _; 
j=0 


= (—Ip—1;B —InB?- - - 1; B')X, . (j > &) 


Therefore 
Ip =-1 
rg =-@Q) 
In =-a2 
I; =0 for j 2 3. (5.72) 


5.2.3.3 Stationality of AR(2). Returning to the general solution of X;, we must con- 
sider the behavior of the complementary function or the solution of the homogeneous 
Eq. 5.60 that is in the shape of 


Ay Mj +A 13. 
This complementary function represents the free oscillation of a forced oscillation system 


which should die out over time t and become asymptotically stationary. For this condi- 
tion it is necessary that 


Wyl<l, ol <1. 
This means that the quadrant, as expressed by Eq. 5.65 equated to zero, 
flZ) =Z?+a;Z+a,=0 (5.73) 


should have roots less than 1 in their absolute values. Therefore, {Z) = 0 must have 
roots inside the unit circle. These roots of f(Z) are the reciprocals of the roots of 


Q(Z) =1+a,Z+aZ’. (5.74) 


Accordingly, @(Z) must have roots outside the unit circle in order for AR(2) to be asymp- 
totically stationary. 
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FD =O, ie., 


Z+a;Z+a,=0 


—-a, + Vai —4a, 


has two roots 


11,2 = 5 ‘ (5.75) 
and these two roots £;, {2 Satisfy Eq. 5.67. Therefore, from the condition 
yl <1, Wl <1, 
1>a,>-1 (5.76) 
and, as “, <> < 1, 
(1 —p2) < 1p. 
Thus My +h2-H1 W2 <1 
and —a,;-a2< 1 
a;+a,>-1, (a,>-a;-1) (3.77) 
andas w,;>-—1, —w,<1, -1<u2<1 
— (1 +2) < (1 + p22). 
Thus — (M1 + 2) — Myla < 1 
and @j—a,<1, (@2>4a;-1). (5.78) 


Therefore, as shown in Fig. 5.15, on a 0 — a}, a2 plane, the region inside of A ABC is the 
stable zone for a; and a2. This stable zone is divided into two subzones. 


1. When Eq. 5.73 has two real roots or coincident equal roots, 
1 
a? > 4a, or a < qu (5.79) 


This is subzone [ I ] in Fig. 5.16. 


2. When Eq. 5.73 has unequal complex roots, 
1 
a?<4a2, or a> reat (5.80) 


This is subzone [ II ] in Fig. 5.16. 
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Fig. 5.16. Stable Subzones [1], [ I]; @), @, @) @).- 


5.2.3.4 Autocovariance of AR(2). From Eq. 5.59 
X; + Q;Xp-4 + a7X1-2 = Ee; 


assuming E[X,] = 0, 


0 
{Ele,- E+] = d(r)oZ= . aay 


0(r) is Kronecker’s delta function. 
From Eq. 5.59 


0 rz0 


E[X,-+-€:] = 
al oo r= 0 (5.81) 


E[X, X,--] + Q,E[X-1 Xi] + a2 E[X-2- Xr] = Elles: Xi). 
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Therefore, when 


r=0 : R(O)+a;R(1)+aR(2)=02 (5.82) 
r=1 : R(1)+a,R(0)+ a2R(1)=0 (5.83) 
r=2 : R(2)+a;R(1)+a2R(0) =0 (5.84) 
r2=2_ generally R(r)+a,R(r—1) + a2R(r —2) = 0. (5.85) 
From Eq. 5.83 
R(1)(1 + a2) = — a4R(0) = —ayo2, 
RO) = 03, 
and therefore 
R(1) -( ml a. (5.86) 
1+a2 


Inserting this value into Eq. 5.84 gives 


ROS ee (5.87) 
l+a2 


Substituting R(1), R(2) into Eq. 5.82, gives 


a (1+ ay)02 


eee (5.88) 
(1 —az2)(1 —a) + a2)(1 + ay + a2) 


This equation connects o? witha. From Eq. 5.86 


I Ge) a ea (5.89) 
(1 —a2)(1 — a, + a2)(1 +a; +. a2) 
From Eq. 5.87 
2 
R(2) = + aj —a2(1 + a2) £2 
1+a2 
+a} —as—ay 


ee Ue a ee ee 
5 Ce Catern Carn (5.90) 


Equation 5.85 is a homogeneous equation with the same coefficient as the homogeneous 
Eq. 5.60. Its solution is in the shape of 
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R(r) = By + Bus, (5.91) 


L1, Mz being the roots of Eq. 5.73, Z) = Z*+a,Z +a =0. Bj, Bz are constants and are 
determined by the boundary conditions, as 


R(0)=02, R(1)= (a+ a2] o2. 


-—aj 2 
1l+a) 


Thus B,+Bz=02 and By; + Boz = 


2 
Therefore B,= Syl Bs a oe (5.92) 
(41 — 2) + ip2) 


2 Ou nae, 


= x: (5.93) 
(U2 —1)(1 + Wifl2) 
po) i 2 
Then R(r)= US LI i pe i o2. (5.94) 
(41 —M2)(1 + 412) (41 —M2)(1 + W142) 
If we need the expressiono¢ from Eq. 5.88 
(EN Ck SOSA 
1 +12 2 
pp eles eS ale 9 (5.95) 
(1 -wia(1—wi)(1 — 3)“ 
then 
R(r) = on ON eA 2 ae EA) Sienna ae eed ame a2. (5.96) 


(41 =n) wD — wea) “1” Gy — pad — wD — per) 
This equation is in the form of R(r) = By 4} +B by. 


There are a variety of cases in which, depending on the sign of 1), 42, the autoco- 
variance function appears different in its tendency. 


1. Whena2 < a?/4 (subzone [1] in Fig. 5.16), 41,442 are real, and 


a. if a, <0, aa>0 (region (1) in Fig. 5.16), “4; >0, uw2>0 
and R(r) stays positive as in Fig. 5.17(A), 

b. if a; <0, a2 <0 (region @) in Fig. 5.16), then 42), 42 have opposite 
signs, 
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c. if a) >0, a2 >0 (region (2) in Fig. 5.16), then <0, p< 0 and in 
this case R(r) alternate signs by r as in Fig. 5.17(B), and 
d. if aj >0, a) <0 (region @) in Fig. 5.16), then 44),42 have opposite 


signs. 
Therefore, for a and c, the autocovariance function shows the patterns (A) and (B) 
in Fig. 5.17, respectively. For b and d, the shape will be (A) or (B), depending on the 


size Of 1,2. 


Ry) of AR(2);a,>% 
(A): a, (a1 < 0,a2 > 0) (B) : ¢, (a, >0,a2 >0) 
Fig. 5.17. A(n) of AR(2); a2 <a?2/4. 


2. When a> > a7/4, (subzone [II] in Fig. 5.16), ;,2 are complex and conjugate to 
each other, and from Eqs. 5.67 and 5.75 


My M2 =a, fy =Vane® = py = Vane"? (5.97) 

and—a@, =; +42, which satisfies 
-a,= 2 Vax coso. (5.98) 
Thus coso = —a,/2 Jaz (5.98’) 
M1 —-b2=2 Jazisinoa. (5.99) 


and 
Using the relations of Eqs. 5.97 to 5.99 in Eq. 5.94 gives 


sin(r + 1) d—azsin(r—1)o 
an ea Dre eee 


(1 + a2) sino 
Here again setting 
l+a 
tang = | ——~ ] tana, (5.100) 


gives 
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Rir)= ail? Sito) a2. 
sin 0 


Equation 5.101 shows that R(r) has the form of a damping periodic function with period 
2x/o and damping a2. 
From Eq. 5.98’ 


(5.101) 


Gz cos{—a;/2 a3) (5.98”’) 
SO a2 determines the damping and a,/ Vay determines the period (see Fig. 5.18). 


fa) 


Fig. 5.18. A(r) of AR(2);a2 >a?/4. 


As an extreme case of stationality, when a2 = 1, from Eq. 5.1000 = 7/2. 
Therefore from Eq. 5.101 


R(r) = cosra oz. (5.102) 


This is an undamped cosine curve and shows that R(r) does not decay with increasing r 
but continues to oscillate in this case. 


5.2.3.5 Estimation of a\, a2. From Eqs. 5.83 and 5.84 


a,R(0) + a2R(1) = —R(1) 
a,R(1) + a2R(O) = — R(2) 


RQ) R(1) RO) R(1) 
~1R(2) R(O) ~1RQ) R(2) 

Thus Gi =) ay a Gy= a ae ae, (5.103) 
RO) R(1) RO) RQ) 
R(1) RO) R(1) RO) 


or 
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, _ R(1) R(2)—R(O) RA) 

2s ye EN ee 
R*(0) —R*(1) 

. _ R°(1)—R(O)R(2) 

TPO Ska) (5.103’) 


’ 


By solving the linear equation on dj, 22, we can easily determine the 4}, @> from the 
autocorrelation R(0), R(1), and R(2). Equations 5.83 and 5.84 can be written in vector 
form as 


R24 =-r, (5.104) 
where 
in ee (5.105) 
R(1) RO) 
a=[d),do’, r=(R(),R(2)]'. (5.106) 
ee 5.107 
therefore a = ( ) 


Thus & = [4;,42]' can be determined from r =[R(1),R(2)]' and 


R(O) RO) 


R2= 1 a) RO) | 


namely, from R(0), R(1) and R(2) . 
After a}, G2 are obtained, a2 can be estimated from Eq. 5.82, 
G2 = R(O) + 4,R(1)— 42R(2). (5.108) 


5.2.3.6 Spectrum of AR(2). From Eq. 5.59 for AR(2) 
X; + a)X;_1 + 22X12 = €;, 
Xe. + Q4Xj-y7-1 + 2X j-+-2 = Err . (5.109) 


Taking the product of both sides of these two equations, respectively, and then taking 
expectation of each term, 
E[X,-X,,]+af E[X1-Xi-1) +f ElX2°Xe+2] 


+a, E[X,- X;,-1] + aya E[X1 -X,,_5] + a2 E[X: X,_,_2] 


+a, E[X;, X-1]+a@3a2 E[X1~-1X1-2] + a2 E[X-X1-2] = Eles: 1+). 
(5.110) 


Therefore, for r = 0, 
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RAr){1 +a;+ aj + a[Rr + 1) + Rr—1)] +.0,0[R fr +1) +Rlr-1)] 


+ a2[R(r+2)+RAr—2)] = Re (r) (5.110’) 
According to the relation between autocovariance and the spectrum, 
=i & jets 
Se + let” = Rr F [jetorF)). pFiw (5.111) 
I Se 
=15(@)) ewe 


Then, transforming both sides of Eq. 5.110’ by Fourier transformation and, taking into 
account Eq. 5.111, 


s(a1 + aj +as+ (a, + ajan\(e™ + &) + ane + | 


1 
= 5<() Seti (5.112) 
1 
sw)|1 + aye + ane? 2 So, (5.112’) 
27 
Thus Sx(@) oe (5.113) 
u = ; : ’ j 
2 r +ajeo@ + ane 2 
from the function a(Z) of Eq. 5.74 
2 
(o} 
sw) = ——————-,, (5.114) 
2| ace) |? 
or 
2 
to} 
$0) = 
2n {a + ajas + 2(a) + a,a7) cos w + 2a) cos 2«)| 
oe 
5 Sg (5.115) 
27 (1 —a2)* + aj + 2a;(1 + a2) cos w + 4a2 cosw) 
Substituting 0? for a2 in Eq. 5.88 gives 
1 —a)[(1 + a2)* —a? Jo? 
S,(w) = CSO aaaaudee (5.115°) 


2m(1 + a){(1 —ay)?* + a;+ 2a;(1 + a2) cosw + 4a2 cos? | 


ds(w) is 


From this expression, and setting 0, 
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a,(1 + a2) 


when a2 >0 and re ety < 1, the spectrum has a peak at 
a2 
-1 a,(1 +a,) 
Oo= SG el Seen ZANE 5.116 
poor aa] sn 
a\(1 + a2) 
When a2 < 0 and reac ee < 1, the spectrum has a trough at the same wo as shown in 
2 
Fig. 5.19. 


a, <0 
USCA eo en(tiesca 
4a, 4a, 


Fig. 5.19. s(w) of AR(2). 
5.23.7 Example of AR(2). In Fig. 5.20, an example of the AR(2) process for a; = 
-0.5, a2= 0.8,02 = 1.0, expressed as X,—0.5 X,_; + 0.8 X;2=€,, was generated over t= 1 


to 600, using the same pure random process €, as was generated in Fig. 5.4. A part of this 


process (t = 100 to 250) is also shown with the time axis extended. Its readings are listed 
in Appendix Al as Table A1.3; pp. 251, 254, and 255. 
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1 S 
Ne eee ees or ie ene Sh eee OT ee sn eee ee 

3 o Ss} 3 8 cS) 8 8 re) 8 
ee * 2. “ wp & ‘ S x 6 


Figure 5.21a shows the theoretical autocorrelation coefficient @(r) = R(r)/R(O) of 
this generated AR(2) process, calculated by Eq. 5.90, using the values of coefficients 


a; =-0.5, a7 = 0.8, and the variance a? = 1.0. Part b of the same figure shows the 


estimated 0(r) = R(r)/R(O), actually calculated from the generated process. An AR(n) 
model was fitted and its order was searched by AIC criteria and found to be n = 2. Then 
parameters were estimated by the method described in this section by Eqs. 5.103’ and 
5.108. The results were 


a; =—0.54156, d> = 0.81838, anda2= 1.04702, 


which are close to the values actually used in generating the process. The autocorrelation 
coefficients were estimated from Eq. 5.90 using these values just as were the theoretical 
values. The results were quite similar and very close to the theoretical values in 

Fig. 5.21a, so again the drawings were omitted. 


-1.0 j T T T 1 
0.00 7.50 15.00 22.50 30.00 37.50 45.00 52.50 60.00 
7T 
Fig. 5.21a. Theoretical. 
Fig. 5.21. Autocorrelation coefficient for AR(2) process 
X,-0.5 X,_, + 0.8 Xo =€1, Er: N[O, 1]. 
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1 


1 | | 1 1 if 1 | 
0.00 7.50 15.00 22.50 30.00 37.50 45.00 52.50 60.00 


3T 
Fig. 5.21b. Estimated. 
Fig. 5.21. (Continued) 


Figure 5.22 shows the spectra, Fig. 5.22a the theoretical one given by Eq. 5.115, 
Fig. 5.22b the estimated one from the fitted model AR(2), and Fig. 5.22c the one esti- 
mated by the nonparametric method from the generated process (Fig. 5.20), using the 
same maximum lag number (60) and Hanning window as for Fig. 5.5c and Fig. 5.12c. 
The a and b spectra look very similar to each other and in this case c also looks similar to 
aand b. In Fig. 5.22c, the 90% level of confidence interval of this estimate based on 


the y” —distribution of the equivalent degree of freedom (here approximately equal to 27) 
as is explained in Section 2.5.5 is shown by vertical lines for reference. 
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Fig. 5.22a. Theoretical AR(2). 


35.00- 
ra 
dé, =—0.54156 
25.005 G2= 0.81838 
o2= 1.04702 


Fig. 5.22b. Estimated by AR model fitting as optimum AR(2). 


Fig. 5.22. Spectrum for AR(2) process X,-—0.5 X,,+0.8 X.=€), €,: NO, 1]. 
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Fig. 5.22c. Estimated by nonparametric method. 
Fig. 5.22. (Continued) 


5.2.4 Second Order Autoregressive First Order Moving Average 
Process, ARMA(2.1) 


More generally for AR(2), when the residual series €; of X;, obtained by subtracting 
the part which is linearly dependent on X;_1, 


6 =X,—-a, X:-1 (5.117) 
was correlated not only with X,_2 as for AR(2) but also with €,_; as shown in Fig. 5.23. 
€t € 


t 


(a) Et VS. Xt-2. (b) et vS. € 44 
Fig. 5.23. Characters of €: for ARMA(2.1). 
Then 


6 = a4 X24+ dE +& « (5.118) 
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€, is now a purely uncorrelated random process, 


Bles-eul=| iit (5.119) 


From Eqs. 5.117 and 5.118 


X,— a1 X11 — 43 X-2 = DE-1 + €; 
this equation is expressed generally, changing — a,’ toa, and—ay’ toa, as 


X, + A,X; + A2X1_-2 = Dj € 1 + €;. (5.120) 
When the process X; is expressed by Eq. 5.120, where aj, a2, and b, are constants and 
€,1S arandom variable, this process X, is called a second order autoregressive first order 
moving average process, ARMA(2.1). 
Equation 5.120 can be written as 


€,= X;+ a)X;_) + €2X;9 — dD €;-} (5.120”) 
This expression shows that, in order to compute the present value of€,, we neede€,_ , and 
when we compute €, recursively, we need the preceding values of €,_; starting from the 
beginning €;2, €;3- - -. This situation is different from AR(1) and AR(2) as shown by 


Eq. 5.7 and Eq. 5.59 for which we do not need any preceding values of €,.;. This makes 


the estimation of the ARMA(2.1) model much more difficult than that of the AR(1) and 
AR(2) models. This difficulty can also be shown as follows. 


From Eq. 5.120’ 


€y-1 = Xj + a1 Xp2 + 2X,_3 — Dy E_2. (5.121) 
Inserting this into Eq. 5.120 gives 


X= — ,Xj_-1 — A2Xj_2 + bilXi1 + a;X;_2 + a2X 1-3 — Di€ 2} Gi 


X= (— a + by)X1 + (a2 + ayb1) Xo + a2b1X,3-Dier2+€r- (5.122) 
Here €;_ 1s still included, so it should be expressed in the same way as Eq. 5.121 by X;_2, 


X-3: - + ande,3, and soon. However, even with Eq. 5.122, when the dependence of X, 
is expressed in terms of past X,, the equation is nonlinear in the unknown parameters a), 
a2, and by. As a result, the regression becomes nonlinear and requires a nonlinear least 
squares method for estimation, which is different from the case of AR(1), AR(2) or the 
general AR model as will be shown in Section 5.4, where all coefficients can be obtained 
by the linear least squares method. The estimation of a), a2, b; for ARMA(2.1) will be 
shown later. 

5.2.4.1 Green’s Function for ARMA(2.1). Equation 5.120 is expressed, using the 
backward shift operator B, as 


(1 + a,B + aB®)X, = (1+ bi B)e;. (5.123) 
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Then 
(1 + b,B) 
Da Game Raa ea eet 
(1 + a,B + a7B~) 


aM (1 + b;B) é 
~ (1=yB)(1— 2B) 
o (1+) 1 _ lite) 1 : 
(GR) 0g) Taal” 


| 
Ms 
eS 
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+ 
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ss 
ty 
+ 
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Mm 
el 


M2 — 


Green’s function is 


(5.124) 


(5.125) 


When 5; = 0, then Green’s function will be that of the AR(2) already given as Eq. 5.70. 
Green’s function can also be easily derived as a solution of the homogeneous equation, 


Eq. 5.120, 
(1 + a,B + a>B*)X, = 0 


with the initial conditions as follows. 
Substituting 


=0 


is 2) ioe) 
X;= ») Gj €1j = b> G; ake 
j=0 
into Eq. 5.123 gives 


(1 + a,B + apB?) p> Gj | E, = (14+ D,B)e, 
ar 


(1 + a;B + a2B*)(Go + GB + GpB? + G3B3 + - - Je, = (14+ d,B)€,. 


Comparing the coefficients of B on both sides of Eq. 5.127 shows 
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(5.126) 


(5.127) 


Get (5.128) 


Go=1 
Goa, + G = by ae SEG (5.129) 
Goa2 + a1G; + G2 = 0 G2 = —a,(bj—a)— a2 (5.130) 


Gj =—a1G;1-a2Gji2, j = 2. (5.131) 


Egs. 5.128 — 5.130 are the initial conditions. 

The solution of the homogeneous Eq. 5.126 is the solution of an nth order differen- 
tial equation, that is, a linear combination of exponential functions, u/ w being the 
characteristic roots, i.e., the roots of the characteristic equation f(Z). Equation 5.73 in this 
case is of second order. 


Therefore for this case, 


Gj = 81 Mi +e wh. (5.132) 
Using the initial conditions given by Eqs. 5.128 and 5.129, 
8it+g2=1 (5.133) 
Sill + 822 = Mi +2 + dr, (5.134) 
where 
(41 + M2 = —a)}). 
Solving gives 
_Mitdi 
Hi-#2 (5.135) 
_ Hatbi (5.136) 
(Lira 
2-1 


Inserting these values into Eq. 5.132, gives the same expression of Green’s function Eq. 
5.125 for ARMA(2.1). As will be discussed in some detail in the next section, in order 
that the original process X;, which can be considered as an output of the input €, , be sta- 
tionary, 4; 2 should be less than 1 in absolute value, | “4; | <1, |“21< 1, wherew 1,2 
are the roots of the characteristic Eq. 5.73. 


fiZ) =Z* +a\Z+ ay =0 


-a\ Ee ¥a?—4a, 


41, Mee aa 2 


for “1 +M2 =—a, and yz = a2. When a?—4a, <0, then 4, “, are complex and con- 


jugate to each other as 
SSG Vv 4a2—a? 
Ee ie ee 


(5.137) 


(5.137) 


siz 


When this was expressed as 


=yer'? 
iesil us ay 4a, -a?)? ae 
7 ial SAI 9 Nae) a = va? 
_1 ¥4a2—a@ ee 5c 2D 


Then inserting Eq. 5.138 into Eq. 5.125 gives 


LAP b ee 
Gy= | O=P Vly cio) | PEt Ny eiey 
Hi-f2 H2-f1 


J ee i 
7 {uy + bie — ua + bye} 


~ (1-2) 
y J COS Jw a= 251 sin jo 
v 4a2 — a? 
Therefore, if putting 
Qi 2b, oh iw, 
V4a9 —ay 


Gj=y/V1+U*cosGw +0) 


de a,—2b, 


where 6 =tan}¢ —————— }. 
y 4a2—a? 


Now let (1+ U? = V again, 
G;=y/Vcos(jw +6) 
= Veli®. pidyi4 VeliVe Oyj 
=i y/+80 yi. 
Then 


21,22= Vex jiMerid = Iglet? Go +9), 
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(5.138) 


(5.139) 


(5.140) 


(5.141) 


(5.142) 


(5.143) 


(5.144) 


(5.145) 


(5.146) 


Therefore Igl=V, (5.147) 
Arg[g] = jw +o. (5.148) 


From Eq. 5.143 G; can be expressed as a damping cosine curve with damping y = Vay 
determined by a> and the period by 


20 _ 27 
SS == =F 
® cos! 
2Vap 
After a2 was fixed, period is decided by a, and phase that is 6 different from that of 


a,—2b, 


| 4a2-a? 


11,2 by 6 = tan™! , as is shown in Fig. 5.24. 


Fig. 5.24. G for ARMA(2.1). 


Green’s function and the autocovariance function are closely related as expected, because 
using Green’s function, 


R(r) = E[xX, X,-r] 


= oc > G; Gjsr- (5.149) 


5.2.4.2 Inverse Function of ARMA(2.1). In Eq. 5.123 


(1 +a, B + a2 B*)X, = (1+, B)€;. 
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If we express €, by an inverse function as 
€,= > -Ij Xj = (Clo-B- IB? ---- 1; B+), 
j=0 
then Eq. 5.123 becomes (1 + a)B + a2B”)X, = (1 +b)B)(—1,B — nB? —- -- I; B!- + -)X,. 


Comparing the coefficients of the powers of B gives 


“Jai (5.150) 
a,=b,-l, 
G7 Dilla 

ee (5.151) 
0=—-J;—by;1,j23 
or 

I, = b,-a, 
Iz = —a2—b1(b; —a)) (5.151’) 


Tj = — byl;-1, j=3- 
The last equation of Eq. 5.151’ is (1 + b; B) J; =O for j = 3. 
Therefore T;=—b, Ij. (5.152) 


This Eq. 5.152 is the recursive equation for Jj. 

5.2.4.3 Stationality and Invertibility of ARMA(2.1). The stationality of ARMA(2.1) 
depends on the convergence of the free oscillation that comes from the complementary 
function or the solution of the homogeneous equation. The homogeneous equation of 
ARMA(2.1) is the same as the homogeneous equation of AR(2), Eq. 5.60, which is 

X; + a1Xi-1 + a2X 1-2 = 0. 

Accordingly, the relations required of the coefficients a, and az for the ARMA(2.1) pro- 
cess to be stationary are the same as for AR(2), as has already been shown in Section 
5.2.3 and in Figs. 5.15 and 5.16. 


On the other hand, the invertibility of ARMA(2.1) depends on the boundedness 


of €;,€; = Sy -lj E,_;> when j — © and from the relation derived as Eq. 5.152, 
j=0 
J; = — b; J;-, \bl should be less than 1. This requirement corresponds to the character 
required for a, of AR(1), la;| < 1, so that the process AR(1) is stationary when 
Gj = a1 G-1. 
5.2.4.4 Autocovariance Function for ARMA(2.1). From Eq. 5.120 


X, = — A1X;-1 — A2X,-2 + €,+ Di E11: 
Therefore, multiplying both sides by X,_, and taking the expected values, 
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E[X, - X;--] = — a)E[X,-1 -X-,]—a2 El X-2-X,-+] 
+ Ele, X,,] + b)Ele 1 X--]. (5.153) 
Another way, by Green’s function, 
X;= Sy Gale) =i Goer Gir Gri Gon eo mom G keer n 
y=0 
where G ; is expressed by Eqs. 5.128 to 5.131. Accordingly, 
Mees (Cy GG ens Gagne? o 0G Epa icre ¢ © 
= € 7+ (by — 41 )€nr-1 + [-42—41(b1—41)} Enr2+- - - 
+ [- ayGi2 — 4G} Gee itiigis (5.154) 
Using these relations from Eq. 5.153 gives 
r=0 when R(0) =—a,R(1)—a2R(2)+02+(b1-a) bio2 ~~ (5.155) 
r=1 when R(1) =—a,R(0)—a2R(1) + by0?2 (5.156) 
r 22 when R(r) = —a)R(r —1)—anR(r—2). Galsy)) 


In order to express the values of R(r) recursively, we have to solve for R(O) and 


R(1), using Eqs. 5.155 and 5.156 after substituting R(2) from Eq. 5.157. Then Eqs. 5.155 
and 5.156 will be 


(1 — a2) RO) +.a;(1 —a2)R(1) = (1 —ayb, +b?) 2 (5.158) 
a,R(0) + (1 + a2)R(1) = byo2. (5.159) 


Therefore 


(1—a,b, +b?) a,(1—-a2) 


b; (1 + a2) 
RO) = ———qqw— os ( 5.160) 
(1—a#) a,(1—a2) 
Qa (1 + a>) 


(1 —a#) (1 —a1b; +b?) 


aj by 
R() = ———————_- 02, (5.161) 
(1—a3) a,(1—a) 
Qa, (1 + a2) 
R(r) = —a\R(r—1)-a2R(r-2) or = 2, (5.162) 
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e(0)=1 (5.163) 
(1—a?) (1—a,b, +b?) 


(a) (By) 
OD). —————— (5.164) 
(1—a,b}+b?) a(1—ap) 
(b1) (1 +a) 
o(r) =—ayo(r—1)—-ap9o(r-2) r=. (5.165) 


These are the autocovariance functions of ARMA(2.1). As was done for AR(2) in 
Section 5.2.3, we can get the same results as the solution of the homogeneous equation 
because Eq. 5.157 is the homogeneous equation with the same coefficient as the homoge- 
neous Eq. 5.126 or Eq. 5.60 of AR(2) expressed by Eq. 5.59. 


Its solution is in the shape of 
R(r) = By My t+ Bz by, (5.166) 
1,2 being the roots of characteristic equation 
fZ=Z2+a,Z+a,=0 (5.167) 
or 
Mi +2 =—-a 
Mi2 = 4. 
B, and Bare constants and are determined by the boundary conditions 
B, + Bz = R(O) 
Bi + Boy = R(1). 


After algebraic manipulation, 


&1 §2 
Bi = 081 Lan 
| Ga CSD) 
(5.167) 
82 81 
B= of @ + —_—————— 
ee 1nd) (l-m1 be) 
or, using the relations of Eqs. 5.135 and 5.136, 
+b +b +b 
Bile oe ew Gece en aan (5.168) 


(uy - me)? | (-“2) 1 -py wo 
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Bee. Ho2t+b) Jeotb) Mithi 


= (5.169) 
“(i -wey | 1-wF 1 -py 
and 
2 D, 
2 
l—-uy 1l-wy 1-fip2 
2 2 
ig ee Ce. Laney , 2+ bit Pi)\ 65 170) 
(@1-H2)) | A-#y) (1-47) 1—p1f2 
_ 9 fh (uy +b)? (uy + bi) + bi) 
1) S © ee ey ee 
(41 — 2) (1-47) 1-Myl2 
2 
ace L2 : ey) — Gt bia + 51) (5.171) 
(4 — 2) (1-45) 1-12 


Using the B), Bz in Eq. 5.166, we get the general expression for R(r). 


The autocovariance function can be expressed using Green’s function as Eq. 5.149. 
Then, starting from the expression of Green’s function 


Gj = 81 Mit 82 M3, 


the same results can be derived for R(r), although the manipulations will not be shown 
here. 


52.4.5 Estimations of a,, a2, b,, and oz of ARMA(2.1). From Eq. 5.157 
R(r) = —a,R(r—- 1) —a2R(r - 2) = 2. 


Therefore, for estimation, with the sample values of autocovariances, 


R(2) = —4,R(1) — 42R(O) (5.172) 


R(3) = —4,R(2) —4>R(A). (5.173) 
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Solving these linear equations gives 


R2) —R) 
EE Benne) O10 


ae RO)-ROR2  &@-62 ~ 
—R(2) —R(1) 
SR) eRe) 
OO) EO) Per —Aatey Ber ans 

oe _K@)-RORG) _ oF@ e(1)e3). (5.175) 


A A i D2 A 5 o 7 AQ ays 
“Ray -2@| B@=-RORQ®™” F)-€@ 


RO) = RO) 


After @, and a2 have been determined by and oz can be obtained by solving Eqs. 


5.155 and 5.156. From them, Pandit and Wu** showed that b; can be obtained as the 
solution of a quadratic equation 


b¢+Cb,+1=0 (5.176) 
where 
1+ 410 + 4202 


C=-a,+ A A A ’ 
—@,—(a2—1)Q1 


(5.177) 
although this is quadratic in terms of b, and is not a linear relation. 
5.2.4.6 Spectrum of ARMA(2.1). The expression of ARMA(2.1) is, by Eq. 5.120, 
X, + QyX,1 + 2X19 = E+) Ep} 
or, using the backward shift operator B, by Eq. 5.123, 
a(B)X,; = BiBye: 


(5.120’) 
a(Z) =1+a;Z+a2Z 
B(Z) =1+),Z. 
Therefore 
Xpor + AXj-y-1 + a2Xj-+-2 = Err + Dy€p-y-1 - (5.178) 


Taking the product of Eqs. 5.120 and 5.178 and then taking expected values of each term 
gives 
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E[X, - X17) + a7E[X 1 - X71] + @SE[X 2 X12] 
+ ayE[X, - X;,-1] + @102E[X;_1X 1-7-2] + @2[X, - X12] 
+ ay E[X,_, X;_1] + 02E[X,,-1 - X;_-2] + @2E[X_, - X12] 
= Ele, € nr) + b7E [E11 > €rr1) + DE [E11 - €1] 
+ bi Ele; - €,+], 
Ryu) +a+ a3| + ay[R,(r + 1) + Re(r—1)] + aya9[Ry(r + 1) + Rr — 1) 
+ a2[R(r+ 2) + R,(r—2)] 
= Re(r) {1 +3] + bilRe( +1) + Re(r—1)]. (5.179) 
Here we use the relation shown by Eq. 5.111 
a > Rr elena OS Re = Wears Vere 
21 I 2m I 
= s(w)-e* ©. 
Taking the Fourier transform of both sides of Eq. 5.179 gives 


s{o){1 + at + as +(a,+a\a yer” + el”) + ar(e2 + erin} 


= Se (W){1 + b2+ by (ei? + 2%] 


1 ! 
== o2(1 +b} + bye +e), (5.180) 
1 
(«| +ajeo" +a ey 2= 5o0e [1 + bye 2 (5.181) 
2) —iw | 2 
o¢ Ble 

Therefore SAW) = cance (5.182) 

2n lace”) 


r +b,e@ | 2 


2 
(o} 
— SSS (5.182’) 
2m rl +a, eo” + ane] 2 


2 {1 +b?+ 2b (ec + e'”y| 


21 {1 + a?+ a3+ (a, +41a2)(e + e&) + an(e™ + erin} 


LG 1+b?+2b,cosw 
2a [1+a?+a4+ 2a\(1+a)cosw +2az cos2w ] (5.182”) 
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Here Ar is assumed to be 1, as was mentioned at the beginning of Part II. As a 
result, S,(@) are obtained in the range —-2 <w <2. However, if Ar =~ 1 and the 


spectrum is calculated for —2/Ar < w </At, then the spectrum is easily inverted into 


oe Be)’ 


ae ae) | a (5.183) 


SA@) = At- 


52.4.7 Example of ARMA(2.1) Simulated. Figure 5.25 is an example of a gener- 
ated ARMA(2.1) process over t= 1 to 600, when a = -0.3, a7=+ 0.4, b} =— 0.7, and 


o2 = 1.0, namely by 


X,—0.3 Xie oF 0.4 X12 = €,—0.7 € 7-1 : 


Here €, is the pure random process we generated as AR(0) in Fig. 5.3. Its readings 
are listed in Appendix Al as Table A1.4; pp. 251, 254, and 255. The correlations and 
spectra of this example are shown in Figs. 5.26 and 5.27. Figure 5.26a shows the theoreti- 
cal autocorrelation coefficient o(r) = R(r)/R(O) of this generated ARMA(2.1), calcu— 
lated by Eqs. 5.160, 5.161, 5.162, or 5.163, 5.164, 5.165, using the design values 
of a; =— 0.3, a2 = 0.4, b} =—0.7 anda? = 1.0. Figure 5.26b shows the estimated 


o(r) = R(r) /R(O), calculated from the generated process. 


We can see in Fig. 5.26b that the autocorrelation coefficient at higher lag r continues 
to fluctuate, even after the theoretical coefficient, shown in Fig. 5.26a, has died down and 
converged to zero. On the other hand, the values at low lag numbers look very similar to 
the theoretical values. An ARMA(n,m) process was fitted and its optimum orders n and 
m were searched by AIC criteria as explained in Section 5.5.4, using these autocorrela- 
tions b, and actually were found to be n = 2, m= 1. Then the parameters were estimated 
by the method described in this section as Eqs. 5.174, 5.175, 5.155, and 5.156. The values 
thus obtained are 


Gd, = — 0.37842, dz = 0.40130, b, =— 0.73968, G2= 1.0400 
which are not very close, but pretty close to the values actually used to generate this 


process. The autocorrelation coefficients 6(r) = R(r) /R(0) were then estimated using 


these estimated parameters by Eqs. 5.160, 5.161, and 5.162, just as for the theoretical 
values. The results are similar and very close to the theoretical values shown in 
Fig. 5.26a, so that drawings were again omitted. 
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Fig. 5.26a. Theoretical. 
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Fig. 5.26b. Estimated. 
Fig. 5.26. Autocorrelation coefficient for ARMA(2.1) process (1) 
X,-0.3 Xe + 0.4 Xpe2 =€E;— 0.7€ 1) Er: N([O, 1]. 
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Figure 5.27 shows the spectra, the theoretical one from Eq. 5.182” in Fig. 5.27a, the one 
estimated from the fitted model ARMA(2.1), using the estimated parameters in Fig. 
5.27b, and the one estimated by the nonparametric method from the generated process of 
Fig. 5.25 in Fig. 5.27c, with a maximum lag number of 60 and Hanning window. Spectra 
a and b are similar and close to each other, and c fluctuated as expected. In another trial, 
instead of the ARMA (n,m) model, an AR model was fitted to this generated ARMA(2.1) 
process. The optimum order n was searched by AIC criteria as AR(n), and was found to 
be n = 6. Its parameters 4) to dg and G2 were estimated by the method described in 
Sections 5.4.3 and 5.4.4 as 


@, =—0.35801, dp =—0.64448, a3 =—0.45424, 
G4 =—0.34915, ds =—0.10554, dg =— 0.11502, 
o¢ = 1.03499. 


These values are listed in Table 5.1. 


The value of G2 is very close to the theoretical value of ¢2 = 1.0. The spectrum 
of this fitted AR(6) is shown as Fig. 5.274. It is interesting to find that this spectrum 
Fig. 5.27d is very smooth, but its shape looks like the smoother shape of spectrum 
Fig. 5.27c, the one estimated by the nonparametric method. In Fig. 5.27c, by vertical 


lines, the 90% level of confidence interval of this estimate, based on the y7—distribution 


of the equivalent degree of freedom (here approximately equal to 27) as is explained in 
Section 2.5.5 is shown for reference. 


As another example of the ARMA(2.1) process, the sign of the parameter bj in the 
previous example was changed, and another ARMA(2.1) process was generated, keeping 


a}, 22, anda? the same, 
a,=-0.3, a=+04, bj=+0.7, o2=1.0. 


As for the previous example, its time history (readings are listed in Appendix A1 as Table 
A1.5; pp. 251, 256, and 257), autocorrelation, and spectra are shown as Figs. 5.28, 5.29, 
and 5.30. 


In this example, the AIC criteria gave ARMA(3.1) as the optimum model to be 
fitted, instead of ARMA(2.1), though the difference in AIC is not so large. The estimated 
spectrum of this optimum ARMA(3.1) is shown as Fig. 5.30b, and for reference the 
spectrum of the fitted ARMA(2.1) model is shown as Fig. 5.30e. ARMA(3.1) gave the 
minimum value of AIC and ARMA(2.1) did not. However, the ARMA(2.1) spectrum Fig. 
5.30e looks more like the theoretical spectrum Fig. 5.30a than does that of ARMA(3.1), 
spectrum Fig. 5.30b. Spectrum Fig. 5.30d shows the spectrum of the fitted optimum 
AR(n) model, where n appeared to be 8 by AIC criteria. It is interesting to find that this 
spectrum Fig. 5.30d shows again the same pattern of variation as spectrum Fig. 5.30c 
obtained by the nonparametric method. The estimated values of parameters 


G},4>,- - -dgandG2 are listed in Table 5.1, and G2 = 1.04911 is very close to the 


theoretical value d2= 1.0. 
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5.00 ] a,=-0.3 


Q2= 0.4 
b, =—0.7 
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7 QO 
Fig. 5.27a. Theoretical ARMA process ARMA(2.1). 
5.00 4 a; = —0.37842 
G2= 0.40130 
b, = 0.73968 
2 = 
3.75 | o2= 1.0400 
4 
2.50 1 
1.25 
0.00 
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Fig. 5.27b. Estimated by ARMA model fitting as optimum ARMA(2.1). 


Fig. 5.27. Spectrum for ARMA(2.1) process (1) 
X,-0.3 Xi1+0.4 Xe =€:-0.7€+4,  €+: NO, 1]. 
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6.2575 


Fig. 5.27c. Estimated by nonparametric method. 
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Fig. 5.27d. Estimated by AR model fitting, AR(6). 
Fig. 5.27. (Continued) 
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Fig. 5.29a. Theoretical. 
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Fig. 5.29b. Estimated. 


Fig. 5.29. Autocorrelation coefficient for ARMA(2.1) process (2) 
X,-0.3 X44+0.4 X2=€:+0.7€.,,  €: NO, 1]. 
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6.255 


Fig. 5.30a. Theoretical ARMA process ARMA(2.1). 


Fig. 5.30. Spectrum for ARMA(2.1) process (2) 
X,-0.3 Xe SP 0.4 X-2 = Ert 0.7€ tH1) Er: N[O, 1]. 
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5.004 é, = —0.20995 
4 d>= 0.30378 
d3= 0.15392 
3.755 b,= 0.80692 


1.0440 


Fig. 5.30b. Estimated by ARMA model fitting as optimum, ARMA(3.1). 
Fig. 5.30. (Continued) 
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Fig. 5.30c. Estimated by nonparametric method. 
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Fig. 5.30d. Estimated by AR model fitting, AR(8). 
Fig. 5.30. (Continued) 
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a, =—0.34559 
G2= 0.41494 
b,= 0 


1.0558 


0.37 0.44 0.50 


Fig. 5.30e. Estimated as ARMA(2.1) [not optimum]. 


Fig. 5.30. (Continued) 
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Table 5.1. Results of estimation by model fitting — [l]. 


Readings Estimated as 
Process ARMA (n,m) a Optimum 


AR (0) 0 0 


Estimated Equivalent 


R (N) 


ARMA (0,0) 
AR (1) 


A1.2 — 0.50933 
1.04646 


2 
— 0.54156 
+ 0.81838 
1.04702 


ARMA (0,1) | 2 
AR (2) 


ARMA (0,2) 
ARMA (2,1) | 


[Process (1)] 


a, — 0.35801 
aa— 0.64448 
+ 0.40130 a; — 0.45424 
a,— 0.34915 
— 0.73968 as— 0.10554 


ag— 0.11502 


o2: 1.03419 


ARMA (2,1) 


ARMA (2,1) 2 


NOT 
1 \ OPTIMUM 


0.34559 
0.41494 
0.69856 


[Process (2)] 


a, 1.00947 
a, — 1.08338 
0.09614 
a4— 0.50333 
a; 0.40894 
as — 0.30626 
a, 0.17930 
a; — 0.09768 


a3 


(OPTIMUM) 


0.20995 
0.30378 
0.15392 
0.80692 
1.0440 


ARMA (2,1) o2: 1.04911 
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5.2.5 ARMA(I.1), MA(2) ARMA(I 2) MA(1), and ARMA(2.2) 
525.1 ARMA(I.1). In ARMA(2.1), expressed by Eq. 5.120 
X; + a, Xj) + 02 X-2 =€;+ DE 1-1, 


when a2 = 0, the process is called ARMA(1.1). We can easily get Green’s function, the 
inverse function, the autocovariance, and the spectrum function of ARMA(1.1) by modi- 
fying those functions for ARMA(2.1) by setting a2 = 0. 


525.2 MA(2). When a; =a2=0 in Eq. 5.120 and there exist b2 in addition to bj, 
X,=€;+ by €,1+b2 €p2. (5.184) 
This process is called MA(2). 
In this case Green’s function is the function for expressing X; by MA (~), so to 


speak of Green’s function for a pure moving average process has no meaning. However, 
formally, 


xX = » G; €,_j 
j=0 
and 
Go=1 
G,=b; 
G2 = b2 


Gj=0 j23. 


In this case the inverse function for expressing X, by AR () is meaningful. From Eq. 
5.184, 


X, = (1+ DB + bpB*) €, 


= B(B)E,, (5.184’) 
here 
BZ = 1+ di\Z + b2Z?. (5.185) 
Now supposing the quadrant 
fo(Z) = Z* +b\Z+b2 =0 (5.186) 


has two roots v; andv2, f2(Z) = (Z—v1)(Z—¥v2) 


1 ae 
V1,V2 =5(-01 + Fi 4b2| 6 (5.187) 
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Then {1 - (vy, +)B +7328 Ae x, 
Vi +v2=—b], 
V1 ¥2= bz 
(1—v,B)(1—v9B) €, =X 


ee x 1 
~ B(B) (1+5,B+b2B2) (1—7,B)(1—v>B) 


€; Tt 


; 15 (vi Ute tobe 
VILRUA 


co yi vfs) 1 
= vat Se ees (5.188) 
FEO Vi -—V2 J 


Therefore, if the expression of the inverse function J; is used here as 


é:= >) CI) Xj, 


i=0 
Mina yitl . , : 

Tj=- a J vyi+ Z v3. (5.189) 
Ui vy) v2—-V¥V1 ilo?) 


This relation, as well as the way to derive it, is the same as that used for the derivation of 
Green’s function for AR(2), already shown in Section 5.2.3. For invertibility of MA(2), 


€,; must be bounded byt — t—j, j — ©. Then by the same logic as was used for AR(2), 


lvy1<1, lv21< 1 should stand for MA(2), and the conditions that must be satisfied by 


bj, bz are the same as the conditions that had to be satisfied by a), a2 of AR(2). This con- 
dition is shown in Fig. 5.31, which is similar to Figs. 5.15 and 5.16, where a), a2 are 
replaced by b,, b2. This triangular zone is again subdivided into subzones [I] and [I], 
depending on 


b? = 4b,[I] and b?< 4b,[II]. 
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Subzone [I] is divided into four regions: 
G) b, <0,b)>0 
@) b; <0,b2 <0 
GB) b; > 0,b2>0 
) b1 > 0,b2 <0 


as shown in Fig. 5.31, just similar to Fig. 5.16. 


Fig. 5.31. Invertible Subzones [1] and [II]; @). @), ©), ® for by, bo. 


Xx; =€,;+ by €,1+ b2 € 12 
Xp = Ep-r + D1 € p71 + 52 € 1-7-2. 


Therefore, multiplying both sides of these two equations respectively and taking the ex- 
pected values of each term, 
E[X, -X,+] = Ele, €:-7]+ bf El€et- €r-+-1)+ 3 El€s2€++2] 


+ by{ Ela evr] + Eley: €x1-1]} + bo Ele 2 €ne] + Els: €nr-2)} 


+ bib2|E leet: €rr-2) + Elna €rr-1]}. 


161 


R,{0) = (1 +b? + b3) Re(0) (5.190) 


R,{1) = R,(- 1) = (b1b2 + by) Re (0) (5.191) 
R,{2) = R,(— 2) = bz - R-(0) (5.192) 
Rr) =0 rs & (5.193) 
because 
2 ee 
Ele; €-,]=06,-O2= Wee ee 6, is Kronecker’s delta function. 
r 


? 


R,{r) = 0 for r = 3. This is an interesting characteristic of MA(2). Generally, for 
MA(n),R,(r) = 0 for r = n+1. In the same way as for ARMA(2.1) 


a2 . . p 
s(@) = 1+ bye + bye?” | . (5.194) 


Estimation of bj, b2, and ag is possible by solving the quadratic Eqs. 5.190, 5.191, and 
5.192 for by, bz, andae. 

5.2.5.3 Example of MA(2), MA(1), and ARMA(2.2). As was done for AR(O), AR(1), 
AR(2), and ARMA(2.1) in the preceding sections, an MA(2) model was generated over 
t= 110600 by X,=€,+0.2€,)+ 0.8€,2 as in Fig. 5.32. Fore,, the same pure random 
process N[0, 1] as was generated in Fig. 5.3 in Section 5.2.1 for AR(0) is used. Its read- 
ings are listed in Appendix A1 as Table A1.6; pp. 251, 256, and 257. Figure 5.33a shows 
the theoretical R(r) using the design values b; = + 0.2, b2 = + 0.8, and o2=1.0, and 
Fig. 5.33b the estimated sample autocorrelation R(r). It is interesting to note that theo— 
retically R(r) = 0 forr = 3. 

Figure 5.34a shows the theoretical spectrum given by Eq. 5.194 where b, = + 0.2, 
by = + 0.8, ando2= 1.0, which are all design values, and Fig. 5.34b the spectrum of the 


fitted model given by Eq. 5.194 where b; = 0.18662, bz = 0.78000, and a2 = 1.0816 calcu- 
lated by the method described in Section 5.3.3. Of course again in this example, order 
determination by AIC, to be explained in Section 5.5, gave the proper value of m = 2. 
Figure 5.34c is the spectrum calculated by the nonparametric method. Here again the 
90% level confidence interval is shown by vertical lines for reference. This was originally 
generated as MA(2). However, as will be discussed in Section 5.4, often an AR model is 
fitted generally to the processes taking advantage of the fact that the determination of 
parameters is much easier than for ARMA. 


Figure 5.34d shows the spectrum of the AR model fitted to this process. The order 
of this AR process is n = 13 and is naturally higher than 2 for m as MA(m). This result 
shows that, if this MA(2) is to be approximated by AR(n), AR(13) is the closest model to 
be adopted. Again, it is interesting to note that the spectrum Fig. 5.34d of AR(13) looks 
very similar to the spectrum Fig. 5.34c estimated by the nonparametric method. 


162 


SSS SSS 
— Sane SSS 
= SS 
— —_ 


° re cS) 8 8 8 8 8 
w N o NN 6 to a o ai io 
' ( 


' 1 1 ' 1 1 i 1 
30.00 37.50 45.00 52.50 60.00 
73T 


0.00 7.50 45.00 22.50 


Fig. 5.33a. Theoretical. 
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Fig. 5.33b. Estimated. 


Fig. 5.33. Autocorrelation coefficient for MA(2) process 
X,=€:+0.2€4,+08€.0, €1: N[O, 1]. 
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4 by = 0.2 
aco bp = 0.8 
ao = 1.0 
4 
3.75 4 
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1.25- 
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Fig. 5.34a. Theoretical MA(2). 


6.254 
5.004 b, = 0.18662 
6 = 0.78000 
c| A 
6.2 = 1.0816 


Fig. 5.34b. Estimated by MA model fitting as optimum, MA(2). 


Fig. 5.34. Spectrum for MA(2) process X;=€,+0.2€.,+0.8¢€12,  €,: N[0, 1]. 
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Fig. 5.34c. Estimated by nonparametric method. 


0.00 
0.00 


Fig. 5.34d. Estimated by AR model fitting, AR(13). 
Fig. 5.34. (Continued) 
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Figures 5.35, 5.36, and 5.37 show the same kind of results for a generated MA(1) 
process, X,=€,—0.7€, , for €,: N[0,1], ie., for b} = —0.7, o2= 1.0. Figures 5.38, 
5.39, and 5.40 are also for an MA(1) process, X,; = €,+ 0.7€,_;, ie., for b; = + 0.7, 
o2= 1.0, just changing the sign of b, from —0.7 to + 0.7. 


Figures 5.35 and 5.38 show the time history over tr = 1 to 600. Their readings are 
listed in Appendix Al as Tables A1.7 and A1.8; pp. 251, 258, and 259. Figures 5.36 and 
5.39 show the autocorrelation. Theoretically, R(r) =O for r = 2. 
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Fig. 5.35. Simulated MA(1) process (1) 
X= €;-0.7€,,, Er: N[O, 1]. 
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Fig. 5.36a. Theoretical. 
1.00 
0.504 
a 
Meu 
-1 1 1a ST RS UL TSU Gal UL NGL LLU 
0.00 7.50 15.00 22.50 30.00 37.50 45.00 52.50 60.00 
7T 


Fig. 5.36b. Estimated. 


Fig. 5.36. Autocorrelation coefficient for MA(1) process (1) 
X= €,-0.7€,,, Er: N[O, 1]. 
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Figures 5.37 and 5.40 are the spectra: a the theoretical, b that estimated by model 
fitting as opumum, c that estimated by the nonparametric method, and d the spectrum of 
the fitted AR(n) model. The m appeared to be 4 and 8 for Fig. 5.37 and Fig. 5.40, 
respectively. 


5.00 5 b, =-0.7 
oI oO? = 1.0 
3.75 4 
2.50 4 
1.254 
4 
0.00 - 
0.00 0.06 O12 O19 42025 0.31 0.37 0.44 0.50 
> @ 


Fig. 5.37a. Theoretical MA(1). 


5.005 b, = 0.70294 
4 G2= 1.0478 


Fig. 5.37b. Estimated by MA model fitting as optimum, MA(1). 


Fig. 5.37. Spectrum of MA(1) process (1) 
X= €-0.7€4,, Er: N[O, 1]. 
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Fig. 5.37c. Estimated by nonparametric method. 
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Fig. 5.37d. Estimated by AR model fitting, AR(4). 
Fig. 5.37. (Continued) 
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Fig. 5.39a. Theoretical. 


i} 1 i} 
0.00 7.50 15.00 22.50 30.00 37.50 45.00 52.50 60.00 


7T 
Fig. 5.39b. Estimated. 


Fig. 5.39. Autocorrelation coefficient for MA(1) process (2) 
X= €,+0.7€,,, Er: N[O, 1]. 
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Figure 5.40, the MA(1) models, did not appear to be the optimum but ARMA(1.2) did 
by AlC criteria. Accordingly, the spectrum of ARMA(1.2) fitted as optimum is shown as 
spectrum Fig. 5.40b, and for reference, the spectrum of fitted MA(1) even though it did 
not appear to be optimum is shown as Fig. 5.40e. 


6.25 5 
4 
5.00 4 
1 Bo OY 
Sh) S| oe = 
H 
2.50 4 
1.254 


Fig. 5.40a. Theoretical MA(1). 
Fig. 5.40. Spectrum for MA(1) process (2) X,=€,+0.7€.,,  €: N[0, 1]. 
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6.25 


5.00 d,= 0.80208 
b,= 1.5161 
bo = 0.55205 
3:75 62= 1.0359 


= 
0.00 0.06 0.12 0.19 0.25 0.31 0.37 0.44 0.50 
70 


Fig. 5.40b. Estimated by ARMA model fitting as optimum, ARMA(1.2). 


1.25 


Fig. 5.40c. Estimated by nonparametric method. 
Fig. 5.40. (Continued) 
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Fig. 5.40d. Estimated by AR model fitting, AR(8). 


25 . 

OE b,= 0.74320 
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Fig. 5.40e. Estimate as MA(1) [not optimum]. 
Fig. 5.40. (Continued) 
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The values of parameters by (and bz, a, if any) and of Ge estimated, and also of 


estimated d,---d, of AR(n) models are listed in Table 5.2. By comparing Fig. 5.37 and 


Fig. 5.40, we can find the difference of the pattern of the spectra by the difference of the 
sign of parameter b;. The same tendency appears in the difference between Fig. 5.27 and 
Fig. 5.30. We can recognize this difference also in comparing the original form of the 
generated processes in Fig. 5.35 and Fig. 5.38, and Fig. 5.25 and Fig. 5.28. 


Figures 5.41, 5.42, and 5.43 show the results for a generated ARMA(2.2) model. Its 
time history readings are listed in Appendix Al as Table A1.9; pp. 251 and 260. The fig- 
ures are the same as for the other examples, so no explanation will be given here. The 
estimated values of parameters dp, Bn. andG? are listed in Table 5.2. The optimum m, n 
for a fitted ARMA(n,m) model appeared to be n = 2, m= 2 by AIC cniteria for this case 
too. In this case G2 appeared to be 0.84312, quite different from a2 = 1.0, but we could 
not find the reason or the error for their poor estimation. 
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Fig. 5.41. Simulated ARMA(2.2) process 
X,— 0.5X,, + 0.8X,. = €,+ 0.2€,,+0.8€,2,  €,: N[O, 1]. 
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Fig. 5.42a. Theoretical. 


-1.00 
y 30.00 37.50 45.00 52.50 60.00 


Fig. 5.42b. Estimated. 


Fig. 5.42. Autocorrelation coefficient for ARMA(2.2) process 
X,—0.5X,, +0.8X,. =€;+ 0.2€ +1+0.8€ 0, Er: N[O, 1]. 
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7 dé; = —0.40678 
15.00 4 d2= 0.80245 
6, = 0.25973 
| by = 0.95205 
11.25 62 = 0.84312 


Fig. 5.43b. Estimated by ARMA model fitting as optimum, ARMA(2.2). 


Fig. 5.43. Spectrum for ARMA(2.2) process 
X,—0.5X_1 + O.8Xp2 = € ++ 0.2€,54+0.8€+2,  €1: N{O, 1]. 
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Fig. 5.43d. Estimated by AR model fitting, AR(15). 
Fig. 5.43. (Continued) 
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Fig. No. 


5.34 


5.37 


5.38 


5.40 


5.41 


5.43 


Table 5.2. Results of estimation by model fitting —— [Il]. 


Process ARMA (n,m) Reece 


ARMA (0,2) 
MA (1) 


[Process (1)] 


ARMA (0,1) 
MA (1) 


[Process (2)] 


ARMA (0,1) 


ARMA (2,2) 


ARMA (2,2) 
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Estimated as 
Optimum 


0.18662 


0.78000 


NOT 
OPTIMUM 


(OPTIMUM) 


+ 0.80208 

+ 1.5161 

+ 0.55205 
1.0359 


— 0.40878 


+ 0.80245 


+ 0.25973 


+ 0.95205 


0.84312 


Estimated Equivalent 
AR (N) 


a, +0.374 
ag —0.201 
a, —0.278 


4230 — 0.064 
41, + 0.167 


432 + 0.001 
233 —0.113 


+ 0.646] a3 +0.299 
+ 0.423] a4 +0.234 


2:1.04127 


— 0.71964 
+ 0.49117 


— 0.30087 
+ 0.26673 


— 0.26548 
— 0.18590 


— 0.09790 
+ 0.08703 


ay +0.091 
449 — 0.252 


41, —0.015 
412 — 0.158 


433 + 0.015 
214 + 0.098 


415 +0.105 


5.2.5.4 ARMA(12) Process. The equation for the ARMA(1.2) process is 


X,+a1X)_1 = E;+ by €,1+b2 € +2; (5.195) 
or, using the backward shift operator, 
(1 + a;B)X, = (1+ b1B + b2B7)€;. (5.195’) 


Analogous to ARMA(2.1), the stationality is determined by the left-hand side of this 
equation, i.e., by the autoregression part. On the other hand, the invertibility is deter- 
mined by the character of the right-hand side of Eq. 5.195’. Accordingly, the stationality 
is the same as for AR(1) and the invertibility is the same as for MA(2). Green’s function, 
the inverse function, the autocovariance function, and the spectrum function can be ob- 
tained in the same way as for ARMA(2.1). 


5.3. GENERAL ARMA(n,.m) PROCESS 


53.1 General ARMA(n,m) Process, Its Stationality and Invertibility 
Analogous to the preceding section, when 
X, + ayXe1 + QnXp2t- + OXpn = Ep ty E4452 Exnt- > -+dmErm, 
(5.196) 


the process X, is referred to as the mth order autoregressive, mth order moving average 
process, ARMA(n,m). When the backward shift operator B is used, 


(1+a,B+a,B2+- --+a, B)X,=(1+b,B + by B?+- - -+b,B™)€, (5.196’) 
a(B)X, = BiB, ; (5.196”’) 


where 
Q(Z) =1+a;Z+aZ* +- - -+a,Z" 


BZ) = 14 b1Z + eZ? +--+ BZ”. Cra 
Green’s function is derived from 
= BB) 
X,= 2 Gj Enj= GB)! 3 (5.198) 
j=0 
and the inverse function is derived as 
= a(B) 
€;= —1;, X_;=——-X,, (5.199) 
t 2, j4cj BB) t 


By the same logic as we had for ARMA(2.1), for this process to be stable, the char- 
acteristic function 


f(Z) = Z" +a,Z™ 1 + agZ** ++ + ++. ay (5.200) 
equated to 0, i.e., f;(Z) = 0 must have all its roots #;,42,- - -, inside the unit circle, 


ora(Z)=1+a\Z+ aoZ* +--+-++a,=0 must have all its roots outside the unit circle. 
Conversely, for this process to be invertible, the characteristic function 
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fxXD) =Z7 +bi\Z7" 1 + boZ™ 2 +- - -+b,,=0 (5.201) 
must have all its roots vj, v2- - - V» inside the unit circle or 

BQ =14+5;Z+boZ +- - -+b,,Z” 
must have all its roots outside the unit circle. 


5.3.2 Green’s Function for ARMA(n,m) 
Green’s function G; is defined as 


X,= DY Gj €j= Go €:+ Gi Ent: +Gj enjt- 
j=0 
= (Go+ GiB + GB? +--+ +G; Bi+- + -e;. 
Then Eq. 5.196’ is 
(1 +a,B +a2B?+- - -+a,B")(Go+ GiB + G2B? ++ - -+G; Bi+-- Je, 


=(1+b,B + boB? +--+ -+bm B”E;. (5.202) 
Equating the same order of backward shifting operator B for both sides of Eq. 5.202, 
Go=1 
a 1Go + G, = by 


arGo+ Gia, + G2 = b2 


QmGo ste G1Qm-1 + G2am-2 +---+G= bm. 


apGo + Giax_) + Goay2- - -+Gp=0 fork > m, (5.203) 
or 
Go = 1 
G, =b,-a, 


G2 = bz — (bi — a1)a; — a2 


Gin = bm, - + ¢—Ap. (5.204) 


Explicitly Green’s function can be obtained through the solution of the homogeneous 
equation 


X;+ aX1-1 + a2Xp2++ + an Xin =0 (5.205) 
with initial conditions given by Eq. 5.204, that is, in the form 
Gi= 21 M+ 92 WAt- + -4+8n Uh (5.206) 
where f;, 2° - -, are the eigenvalues or the roots of the characterization equation 


fi(Z) =Z" +.a;Z") + anZ** +- - - a, = 0. 
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Especially when m = n—1, namely for ARMA(n, n — 1), by Pandit and Wu*° 
(uz 1+ byw + - - - +bp-1) 


BO eee 
" (Wi—M)Ui— M2) + + Ui — Me) Ui—Mis dd «+ Hi-Ln) 

US Mor 2 81g (5.207) 

where the denominator is the product of all terms (u;—;) for j= 1,2, - - - n, excluding 


the zero term (u;—;). Equations 5.135 and 5.136 are the special cases j = 1 and 2 for 
n=2. Each real root yu; in Eq. 5.206 provides an exponentally dynamic mode like an 


AR(1) model, whereas a complex conjugate pair of roots as in Eq. 5.137’ gives an expo- 
nentially decaying sinusoidal mode, whose frequency and decay rate can be obtained as 
Egs. 5.140 and 5.139 illustrated in Fig. 5.24 in Section 5.2.4. 


5.3.3 Autocovariance and Spectrum Function of ARMA(n,m) 
For the ARMA(n,m) process, 
X, = — 4X1 —A2Xp-2—+ + ApXtnt+€r+ dy €-1+ 52 €p-2+- > -+dm Erm. 
Here, taking into account the relations 


X,= YG €.j=Go €:+ Gy €-1+ Gz rat: -+GjEpjt+- -- 
Xp = SG Sp pap SO Gorn Go ara Gy see 5 oe (Ge Gra nee % 6 


Ele,- €,-+] = 0, oe ’ 


multiplying both sides of these upper two equations term by term, respectively, and tak- 
ing the expected values gives, 


when r=0, 
R(O) = —a,R(1)—a2(R)(2) —- - -—a,R(n) + (G + biG; + b2G2 +: - - bmGm)oe 
when r= 1, 
R(1) = —a@;R(0) — a2R(1) — a3R(2) —- + -—a,R(n—-1) 
+ (b}Go + boG + b3G2+- > - + DmGm-1)02 
RQ)=-----:: 


when r =m, 
Rim) = —a,R(m—1)—a2R(m— 2) +: - -a, R(m—-n) 


+Dm Go a2 
and 
R(k) =—aiR(k-—1)—aoR(k—2)+- - -+a,R(k—n). k => m+1. (5.208) 


Analogous to the case of ARMA(2.1), R(0) to R(k),k = n+1, can be obtained by solving 
these simultaneous equations. Also, if necessary, we can get the general expression for 
R(r),k = m+1, by generalizing the case of ARMA(2.1). By the same procedure we 


used for ARMA(2.1), the spectrum s(w) for the ARMA(n,m) process is naturally 


183 


obtained as 


o2 pe”) P 
s(w@) = —_————_ (5.209) 
2n Jace) P 
a2 LS Dy 3 yO eo 2 och fh BPO || ¢ 
= = (5.209’) 
| GO EGRET oe 3 atk GPO || @ 
5.3.4 Estimation of aj ... Qn, by . . . bn of ARMA(n,m) 
For starting values of a)- - - G,;, b,: - - b,, we can get a first approximation of the 
estimates from Eq. 5.208. Namely, with the use of n simultaneous equations on 
R(m+1), R(m+2)-+--R(m+n) fork =>m+1 
autoregressive parameters a; , d2,- - - a, can be estimated, although the less reliable val- 


ues of autocorrelation at larger m + n must be used. Then, inserting these values in the 
first (m + 1) equations in Eq. 5.208 on R(O), R(1) - - -R(m), m+ 1 unknown parameters 


by: - - bm and a2 can be estimated theoretically by solving the nonlinear equations on 
these parameters. 


The necessity for solving nonlinear equations can be shown as follows: In the 
expression for ARMA(n,m) 


Xp + QyXp1 t+ a2Xpot: + anXpn = €:t+ dD €-it- > -+dDm€im, 


using recursively 
€ py = Xp + OyXp2 + G2Xp3 t+ ++ An iXin t+ nXtr-1 


— by€,-2— b2€-3—- + > — DmEt-m-1 


X, = — X14 —42X,2° + + —ApXin 
+ bE + 07E 5-9 - - H Dpe em + er, 


X, = (—@1 + by)Xj-1 + — 2 —4b1)X-2 ++ + + + (— An — Gn_-101 Xin — And) Xi-r-1 


+1(b+ = bo )e 29 + (DD Dae nat bee ere (5.210) 


We still have the terms €,2,€,3- - : that should be expressed by X,2, X,3- - - and also 
€;-2,€,3. When the dependence of X; is expressed in terms of past X;, the equation will 


be nonlinear in the unknown parameters a), a2: - - and bj, bo, - - -, because their prod- 
ucts and squares are involved. Thus, the regression becomes nonlinear and requires a 
nonlinear least squares method for estimation. 
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Actually, however, it is troublesome to solve the nonlinear equations, and other ap- 
proximation methods are used. An approximation method introduced by Pandit and Wu*° 
is to use the inverse function, taking advantage of the linearity between the unknown 
coefficient and the inverse function as shown in Eq. 5.150 for ARMA(2.1). In this meth- 
od, we first fit the pure autoregressive model of an appropriate large order AR(&) to this 
ARMA(n,m) process and then use the coefficients of this AR(k) model as an approxima- 
tion for inverse functions at larger values of k. Then from the equation that connects b; 
and J;, we can gradually improve the approximation of b; and a; and finally adjust the 
results from the viewpoint of invertibility. The procedure is rather complicated, and care 
is necessary to get good estimates. In any case estimation of the coefficients for ARMA 
(n,m) is not necessarily easy. 

Estimation of the parameters for the pure autoregressive process AR(n) is more 
straightforward, as we saw for AR(2) in Section 5.2.3 or as will be shown in Section 5.4, 
since it can be done through linear regression. 

In practical applications of the model fitting technique, the pure autoregressive pro- 
cess AR(k) is frequently used as the model to be fitted, although the order k sometimes 
becomes a little larger than n of ARMA(n,m), which should be the actual model to be 
fitted to the process under consideration. We saw this tendency in examples of 
ARMA(2.1), MA(2), MA(1), and ARMA(2. 2) shown in Figs. 5.27, 5.30, 5.34, 5.37, 
5.40, and 5.43. 


5.3.5 Adoption of ARMA(n, n— 1), ARMA(2n, 2n —1) 
As was mentioned in Sections 5.2.2 and 5.2.4, Green’s functions and the autocovari- 
ance functions are in the form for AR(1): Gj =a! =y'; R(r) = Bu! and for ARMA(2.1): 


Gj= 21 uw) + 22 wi; R(r) = Buj + Bz wu}, and show increasing complexity as the order 
increases. For AR(2), as derived in Section 5.2.3, G; and R(r) were of the same type as 
those for ARMA(2.1). However, as was mentioned in Section 5.2.4, ARMA(2.1) is rec- 
ommended by Pandit and Wu* as a more general process that is made up of two expo— 
nentials. The other reason for preferring ARMA(2.1) over AR(2) is the fact that the 
ARMA(2.1) process is more closely related to the system that is governed by the second 
order differential equation, as will be explained in Section 6.3.2. In the same way, for a 
more complex process made up of three exponentials, AR(3.2) is the most general pro- 
cess. Extending this relation, ARMA(n, n—1) is recommended by Pandit and Wu" as the 
most general process when the process is represented by nth order dynamics as 


Gj= er Miter Wht +n Meds RO) = By w+ Bo wht: - -Bybs. 


The reason that n — 1 should be the order of the moving average part is also shown in 
Pandit and Wu.*° Furthermore, as they pointed out, empirical experience indicates that it 
is better to increase n in steps of two and fit ARMA(2n, 2n — 1) models for n = 1, 2, 3. 
Also, if a complex root appears among #4;, and a), - - - are to be real, complex roots 
must appear as conjugate pairs as was shown for ARMA(2.1) in Section 5.2.4. Therefore 
to fitan ARMA(2n, 2n — 1) model is more practical than to fit an odd order model. 
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5.4 GENERAL AUTOREGRESSIVE PROCESS AR(n) 
5.4.1 Adoption of AR(n) Model 


The general autoregressive model 
X,+@)X,1 + a2X2+-- -+a,=€; (5.211) 


is considered to be a special case of the general autoregressive moving average model 
ARMA(n,m) when m = 0. If invertibility is satisfied for the ARMA(n,m) model, 


€,=| > -1B IX, 
j=0 
SO Ii AGI 2G ee 9 ole Gea 8 a, 

This expression means that ARMA(n,m) is transformed into an autoregressive model of 
infinite order AR( ). Practically, if a large enough n is taken, almost all of the ARMA 
(n,m) process can be approximated by an AR(n’) process if we assume invertibility. 

The general character of the AR(n) process will be summarized here, as AR(n) is 
the most common model used in the practical application of model fitting techniques as 
was already mentioned in Section 5.3.4. The estimation of parameters is much easier for 
the AR(n) process than for the MA(m) or AR(n,m) processes. 
5.4.2 Green's Function of AR(n) 

The AR(n) process is expressed as 


X,+ 4X1 +Q2X,2+-- -+a,=€; (5.211) 
(1+a)B +a7B7+- - -+a,B")X, = €, (5.211) 
a(B)X, = €;. (5.211”) 
Here 
Q@(Z) =1+a;Z+a,Z*+- - -a, Z". (5.212) 


Using Green’s function G; gives 


ive} 
X,= ». G; €-j= SG Bi €,=Got+Gy €11+ Gr€p2+: - -+G; €rj 
j=0 


jJ=0 
= (G,+ GiB +G2B* +--+ -+GBi+-- Je, (5.213) 
ee (S282) 
(1 +a,B + a2B trois -+ 7B”) 


Equating the same power of B on both sides, 
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Go=1 


Goa Gr G, =i) 
a2G,+a,G; oP G2 =0 
a3G, + a2G; + a;G2+G3=0 (5.214) 


anGo + a,-1G} ape © oap(Gr, = 0, 


or 
G,=-a 
G2 =-a2+ a? 
G3 = —a3 + a0) + a\(a2 — a?) = — a3 + 2aja2 — a}. (5.215) 
Generally 
(a,B" +--+ -+1)G,=0. 
Explicitly as shown in Section 5.3.2 
Gj= gi Wi ter wht > + Bnbth G216) 
where jl; : - «4, are the eigenvalues or the roots of the characteristic equation equated to 
ZeTO as 
fi@)=Z+4a,z7""+---+a,=0 (5.217) 
and 
n—1 
mS ent (5.218) 
(Ui — 1) (1 — 2° > + i Mi) Ui Hiv) * «+ Ui Hn) 
where the denominator is the product of all terms w;- - -wjfor j=1,2---n, excluding 
the zero term uu; —;, as the special case where b} = b2 = b3 = -= b, = OmEg. 
5.207. 
5.4.3 Autocorrelation and Spectrum Function for AR(n) 
For AR(n) 
Xp = — 1X11 — A2Xp2° + + —GpXint €r. 
Therefore 
xeun =i QXi-7-1 = aX 7-2 — See ee AnXt-r-n +€r,. (5.219) 


Multiplying both sides of these two equations, respectively, taking the expected values 
gives 
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R(O) = —a;R(1) —apR(2)- - -—a, R(n) +a2 
R(1) =—a}R(0) —a@2R(1)- - -—a,R(n-1)- 


(5.220) 


R(k) = —ayR(k—- 1) —a2R(k—2)- - -—a,R(k—n) 
g= Iho o 970, 
Using the backward shift operator B gives 
BIR(k) = R(k-j). (5.221) 
Then the last equation in Eq. 5.220 is 


R= ( >) =a; Bi Jn (5.222) 


j=l 


or 
n 
SG BR = 0. (5.222) 
jJ=0 
Equation 5.222 is very similar to the expression of the homogeneous equation of the 
process itself in Eqs. 5.211’, 5.211” when €,=0 as 


a(B)R(k) = 0. G:222) 
Therefore, as was Eq. 5.166 for AR(2), the general form of R(&) is 
R(k) = By wh + Bo ub+-- +B, uk (Goze) 
Here “),42- : -M, are the eigenvalues, and B,, Bz, - - - B, are constants that can be 


obtained theoretically from the boundary conditions 
R(O)=B,+B2+---+B, 
R(1) = By fi + Bz fat: > + Br Mn 


(5.224) 
The spectrum is, as given by Eq. 5.209, 
2 2 
S(@) = AGN = Uae In ISITE aN a (5.225) 
21 | a(e*)|? 27 | 1+ ajen" tae + + apex |" 
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5.4.4 Estimation of Parameter aj ... a, and u 


In this section, we assume that the order n of the autoregressive process AR(v) is 
already known. Determination of the order n is discussed in the next section. Here the 
Statistical considerations for the estimation are treated in some detail, compared with the 
case for AR(2) in Section 5.2.3. As the most general type of AR(n) process, we assume 
the nonzero mean process to be 


(X;—M) + ay(X-1 —M) + A2(X-2-M) ++ > + + An(Xin—M) = €,. (5.226) 


By analogy with the multivariate regression of X; with X;_,- - - X;_n, with regressive 
coefficients a), a2: - - a, and residual error €; , we can use the least squares method to 


estimate a,- - - dp, #,ande,, that is, to minimize 


N 
OWiai---an)= > ef 


t=n+l 


N 
2 
= Sy {X-w) +aXi1 —bM)+:- “+ On(Xen —H)} > 227) 
t=n+l 


The reason for this lower limit n + 1 is that we do not have observations Xo, 
X1,° +X hah) from which to derive €,,- - -€, . Nis the total number of observations 
of X;. The residual error €, is assumed to be Gaussian and independent of each other. 


Accordingly, the joint probability distribution of €,4;: - -€, 1S 


1 N-n “= er 
20, 
€ 5€ 9) 0).0(s = II e € 
P(En+1,€n+2 N) Jano, pa 
Paw ae 
om 2 
= e — € (5.228) 
Tine.) | 208 © 
Tone) ana = 
=| ——_ ex ,41,42° °° an) (5.229) 
lems. P 22 Qu 2 n 
Here the variables are transformed from (€n41,€n42° ° °€n) tO (Xn41,Xnv2,° ° * Xn) 


using 
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Ens = (Xn41 —H) + 21(Xn—M) + A2(Xn-1—M) > + + An(X1 —) 
€n42 = (Xn42 —[) GP A(Xn41 — pb) ote a(x, —[) BCR OES a,(X2 —L) 


(5.230) 


€n = (Xy—M) + y(Xn_1 —L) + A2(Xn_2 -—M) +> + + + Ap(Xn_n— W). 


From Eq. 5.230 the Jacobian to transform p(E,, 1>€n+2° ° °€N) tO P(Xn+i1,Xn42° * Xn) 
is 


Es! O(En+1 En+2° 01-0 En) =a (5 231) 
O(Xn+1 Xn+2° * Xn) 7 
Therefore from Eq. 5.229, 
1 —n 
XG AG, a) ahs -X, = x 
P( +1 +2 N) J2n0¢ 
i 2 
exp—sq DY [Ki-w) taKer—w) +--+ ann} 
€ t=n+1 


N-n 
1 -1 
exp| —> 5@1,€2° ° -An)} |. 5.232) 


This is actually the conditional joint distribution, given that the initial observations 
X, - + -X, remain fixed at their realization X,- - - X,. Here n is assumed to be 
relatively small compared with the number of observations N. We can then use this 
P(X nai.° * *X ny) as the adequate approximation of likelihood function of 


/1,41,42° - - Gp, given the observations X;, X2- - - Xy. Then the log—likelihood 
function L is 


LW, 41, a2, igi 7 an) = log PXn+1,Xn+25 af ete Xn) 
N 
—(N—-n) 1 2 
= ————_log(2 o2)-—5 ys {(X,—m) + a1(Xa—-p) + . + dn(Xin — H)| : 
20¢ t=n+l 
(5.233) 
Therefore the maximum likelihood estimate for a, - - - a, is to maximize L, i.e., to 
minimize 
a 2 
Ous,41,42° + -a,)= > {&-m) + CX) tl ot an(Xin— 1] . (5.234) 


t=n+l 
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This is the same as adopting the least squares estimate as already mentioned. That is 
obtained from 


) 
O2 La 
Ou 
and 
F) 
ag Fete cen 
0a; 
as 
N 
S. [&-m) + arKer—w) +: + + ayKen—H)} = 0 (5.235) 
t=n+l 
and 
N 
y {(X-w) +a(X-1-M) + - + af(X,j;—M)- > > +a,(Xpn—/) |X j-w = 0. 
ci ec (5.236) 
From Eq. 5.235 
fi oeeeenhenioe sera (5.237) 
1+a,+d.+:--+4, 
where 
oy all Ny 
t=nt+l 


N 
= a el 
When n is small compared to Nj, X; will be close to X = N » X, and Eq. 5.237 will be 


t-1 


pi=X. (5.238) 
From Eq. 5.236 
Ny a a — —_— 
{Op -¥) + bi Kea-H) ++ + GK j-K)- +4, (Kin-B] x 
t=n+l 


{Xj-H}=0, fal --n. 
Here, if we approximate further, under the assumption that & is also small in relation to N, 


then 


N 
DY Xia X\Xj-X) = NRG-8), 
t=n+l 


and Eq. 5.236 becomes 
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R(j)+a,RG-1)+- - -ajR)- - -+a,RG—n) = 0 (5.239) 
PS ilo o om 
From this relation d;,@2,- - -d;- - -d, can be estimated by solving the simultaneous 
equations 
a,R(0) + a>R(1) +: - -a,R(n— 1) =—R(1) 
a,R(1) + aoR(0)+- - -+a,R(n—2) =—R(2) 


(5.240) 
a,R(n—1)+aR(n—2)+- - -+a,R(O) = —R(n). 
This can be transformed, using the vector expression, to 
(a1,02,+ + +a)’ =a@ (5.241) 
{ia Vio <5 Ro} =e, (5.242) 
RO) R()--:R(n-1) 
RG) (RO RO 2) 
R,= |. ne (5.243) 
R(n-1) R(n-2) R(0) 
R,-a=-r. (5.244) 
Therefore 
A -—Tr 
OF aaa (5.245) 
Rn 


Equation 5.239 or 5.240 is identical with the Yule-Walker equations. 


All these relations mean that, if the approximations as given here are introduced, the 
maximum likelihood method can be replaced by the least squares method, and the least 
squares method is reduced to get the solution of the Yule—-Walker equations. Priestley”? 
describes in more detail the precise estimation through a rigorous likelihood function. 
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5.4.5 Estimation of a2 for AR(n) 
N 
oe is estimated as the variance of the residual error €, from Q = yy. EP, 
t=n+l 


Because (N — n) observations are summed and n + 1 parameters (u and na;’s) have been 
estimated, the unbiased estimation is 


1 
a2 A A A A 
— Omaha a 
€ (N—n)—(n a eae 1,42 D) 
: S {x-¥) X.4-X) Jina 
= = + _j- tee et mA ye 
omni al Q\\A;-] AylA; 
=" _[r@) +4 RG ee R(n)| (5.246) 
7 Neanest : ; 
When n <<N, 


G2 = R(0) + a,R(1) + + -+ anR(n) (5.246’) 
This result is the same as we obtained from the first equation of Eq. 5.220. 


5.5 DETERMINATION OF THE ORDER OF THE FITTED MODEL 

As was explained in the preceding sections, the process becomes more complicated 
as the order increases. Thus AR(2) is more complicated than AR(1) and ARMA(3.2) is 
more complicated than ARMA(2.1). Estimation of parameters has been discussed in 
each section, so here only the determination of the order will be discussed. Several 
different methods have been described by Priestley.2*? The MAIC method developed by 
H. Akaike is now considered to be the most reasonable from the point of view of statisti- 
cal considerations and of information theory and is explained and recommended in the 
following section. 


5.5.1 Residual Error Method 
The change of residual error, expressed for example by Eq. 5.246’ for AR(n), is 
investigated. If the order is far smaller than the true order, o2, the residual error, will 


decrease considerably as the order increases from j ton. From there on, the decrease in 
residual error may not be significant but will remain at the same level. Point n is adopted 
as the proper order of the model. 
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Fig. 5.44. Residual error. 


Sometimes, however, it is not easy to determine the order because the rate of change 


does not show clearly by this method. 


5.5.2 Partial Autocorrelation Method 


If a,, denotes the ith autoregressive parameter of an AR() process, then the auto— 


correlation coefficient 9; is from Eq. 5.220, 


Qi = — 4n1Qi-1 — 2720i-2° * 


p12 in 


which gives the Yule-Walker equations, 


Q1 =—4n100—4n201°- 
02 =—4n101—-4n200° - 


On = — GniOn-1 — 4n,Or2°- 


*— Ann Qi-n 
(5.247) 
*— Ann Qr-1 
*— Ann On-2 (6 248) 
*—GAnnQo- 


Then, the last coefficient a,,, of the fitted AR(7) model is called the partial autocorrelation 


Qn, forn = 
Since 0, = 1, from Eq. 5.248, 


—411=01>01' 


1, 2- - - , and will be used*” to suggest the order of the AR model. 


11 
Q1 Q2 ao 
Sipe EE = (5.249) 
1 01 (Gil 
Oxeel 
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Oh | Oe oom cy or 


Cie ein aamen Or=si02 
1 @n1 
Or-1 On-2 On-3°°° Q1 On 
ci we Abt ELA CANE NS SS: (5.249") 


ID Oi) Ore C2 Ot 
Q1 1 @1-°-*@n3 Qn2 


1) @A 
Or-1 Qn2On3°°°OQ1 1 
A plot of 0;'02'- - -@,' against n is called the partial autocorrelation function diagram. 
If the true order was n, 0,’ Or — a; ; Should vanish for k > n and the plot of the partial 
autocorrelation should show the real order 7. 


For this purpose, a recursive method for calculating 9,’ when — dp, are estimated 
from Yule—Walker equations has been given by Durbin? as 


Qn+1oj = Gn j—Ansine1 Gann jis J=l,---n (5.250) 


6(m+1)- > 4, ,6(n+1-j) 
: (5.251) 


Qm+1,n+1 = 


j= 
j=1 


553 Visual Inspection of the Autocorrelation 


As has been shown in the preceding sections, the types of models and their orders 
are well reflected in the pattern of the curve of the autocorrelation function. For example, 
for the MA(m) model, the correlation cut down to 0 after the lag m, and for the AR(7m) 
model, the correlation has the form of damping oscillations. Box and Jenkins*’ have 
developed this method and showed many examples of estimating the order from these 
patterns. However, it requires skill and long experience to be able to estimate the actual 
order correctly by this inspection. 
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55.4 Akaike’s FPE, AIC, and BIC Criteria 


FPE. Akaike“ first proposed computing the final prediction error (FPE) in deter- 
mining the order n of AR(n) as 


N+q: 
FPE(q) = ——1-62 (5.252) 
N-q 


where N = number of observations to which the model is fitted, g = the order of the AR 
model 


62= {R(0) + a:R(1) +- + agk(g)| 


and adopting as n the value of q that minimize the FPE, as in Fig. 5.45. 


FPE (q) 


fe 


n 


Fig. 5.45. FPE vs. q. 


AIC. Later Akaike*!~* proposed a more refined method of minimum AIC 
(Akaike’s Information Criteria) based on information entropy theory. This is a very gen- 
eral concept available for general statistics problems, as an example of curve fitting, that 
is shown in Appendix A-2.*° AIC is defined as 


AIC(q) = (— 2) loge[Max. Likelihood] + 2¢ (5.253) 
where q is the number of parameters. Including G2, n + 1 parameters have to be 


estimated, so 
g=ntl. (5.254) 


The log. [maximized likelihood] is L in Eq. 5.233 or, omitting the constant, 


N-n 


1 
2 

log Oe ~ 202 Q. 

Here Q is defined as the sum of squares and from Eq. 5.246 
R 1 
a2 
Oc = N Q. 
N-n 


2 
Ignoring the second term, which is constant, and inserting it in Eq. 5.253 


Thus pes 


to N 
logde-—@. (5.255) 
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N-n 


AIC(g) = (- 2)| -—log 2 | + 2 


= (N—n)log62+2(n+ 1). (5.256) 


AIC (aq) min. 


Fig. 5.46. AIC(q) vs. g. 


The AIC (q) is plotted against n as in Fig. 5.46, which shows where the AIC (7) has 
a minimum value. This minimum value is called MAIC, the minimum AIC, and this 
method for obtaining that MAIC is called the MAIC method. Akaike*!4? made clear that, 
based on information theory, AIC is a measure of close fimess of approximation = a 
measure of minimum difference: (statistical model—true model). 


So if we express the statistical distribution of the occurrence of the data for the 
model as p(x) and the statistical distribution of the true data derived from the true struc- 


ture of the data, as g(x), to minimize AIC is to minimize 


x 
[= | patog®, 
q(x) 
the so—called Kulback’s*’ information criterion. Also the so-called information entropy 
S(q,p) is defined as S(q,p) =~ —J. So to minimize AIC is to maximize the entropy 


S(q,p). Accordingly, the minimum AIC criteria are also called the maximum entropy 
criteria. 

FPE was introduced to find the order of the AR model. AIC is, however, a very 
general criterion and can be used for ARMA or MA(m) models too. For the ARMA 
(n,m) model, we can use this criterion setting 


AIC(n, m) = (N—m—n)logd2+ 2(n+m+ 1). (5.257) 


Plotting AIC(n,m) over an appropriate grid of n, m we can adopt (n,m) that minimize 
AIC(n,m). 
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Akaike also made it clear that, for large N, FPE, and AIC are equivalent, because 


l+g/N 3 
q/ 62 


log FPE(q) = lo 


= log(1 + g/N) -log(1 —q/N) + log a? 
=logo2+2q/N (for large N). (5.258) 


Therefore AIC(q) = N log[FPE(q)]. 


When N, the number of observations, is large enough, the FPE criterion is the same as the 
AIC(q) criterion. 


BIC(q). Akaike***° also proposed the BIC(q) criterion, based on the Baysian con- 
cept, as a new criterion for determining the order. That is, 


BIC(q) = Nlog 62— (N —q)log(1—q/N)+qlogN 


aloe 
+ qlogsq tI ; (5.259) 
O¢ 


This equation can be modified as follows if we approximate 
((-N -q)log(1—q/N)} = {|-W-9)(-4/M)} =9-@°/N = 4 
for N >> q. Then 


€ 


a2 
BIC(q) = AIC(g) + gqlog N—1)+q oof 5 1 | } 


= [Nlog 2+ 2g] + qdogN)—q+qlogsg| 52-1] +b. (5.260) 
C, 


This relation shows that the difference between BIC(q) and AIC(q) is approximately 
one g of AIC(q) and is replaced with g log N. This replacement has the effect of increas- 
ing the weight attached to the penalty term which takes account of the number of 
parameters in the model. Shibata>® tells us that AIC(q) slightly overestimates and BIC(q) 
slightly underestimates the real q value. 


55.5 Examples of Order Determination through MAIC 


In Sections 5.2.1 to 5.2.5, examples of the synthesized processes AR(0), AR(1), 
AR(2), ARMA(2.1), MA(2), MA(1), and ARMA(2.2) generated by mathematical models 
were shown. When these processes were given as observed data, the orders N and M of 


each process were estimated by the MAIC method and then parameters a; - - - ay; 
b,- - - bm were estimated by the method described at each subsection. The results are 
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summarized in Tables 5.1 and 5.2. From these results, we can find the criteria of MAIC 
that give us the correct values of the order except in a few cases. Estimation of parame- 
ters @}- - - @,3b,- - - b,, and az is rather reasonable in these examples. Tables 5.1 and 
5.2 list the AR(n) model fitted to the original ARMA or MA models. It is interesting that 
the values of N obtained by the MAIC method are somewhat larger than n or m of the 
original processes. These models are the AR models which approximate most closely the 
original ARMA or MA models. We can say these examples gives us good proof that the 
model fitting method, supported by order determination through AIC criteria, is a very 
reasonable and powerful method for analyzing the linear stochastic processes. 
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CHAPTER 6 


MODEL FITTING TO THE RESPONSE OF THE LINEAR DYNAMIC 
SYSTEM TO IRREGULAR INPUT 


6.1 INTRODUCTION 


In Chapter 5, the characteristics, stability, and invertibility of a single linear stochas- 
tic process X; were discussed and it was concluded that AR, MA, or ARMA models of 
appropriate order can usually be fitted to most of the linear stationary time series to 
represent their statistical properties. 


The characteristics of the model fitting techniques is, the author believes, that all of 
these AR, MA, and ARMA models relate the process X; to the pure random process 


€,, although the relations that connect €; to X; are different for the different models. 


For 


AR: X,+@)X;-1 + a2Xp2 +... tQnXpn = €1; 


MA: X,=€; + by€+-1+ b2€-2 . . . tbm€rm; and (6.1) 


ARMA : X, + A1Xj_1 + a2Xp-2 6 0-0 + AnXp-n = Ey+ by€;-1 


+ b2€ 12 GP 60 6 FF bmE tm . 


In all of these models, €, was supposed to be the input to an imaginary system, and 


X, was treated as the output of the same system. All the characteristics of X, are expressed 
in relation to those of the pure random (white) process. 


6.2 RESPONSE SYSTEM WITH FEEDBACK 


6.2.1 One Input / One Output System 


Here we consider the output Y of a real linear system with input X;, usually ex- 
pressed as 


Y; = yy SuXtu + €;. (6.2) 
u=0 


Here g,, is the so—called impulse response function of Y; to X; or the weighting function of 
this linear system to X;. Now suppose we have N observations of the input/output pair 


{X,, Yi}, t=1 to N. 


If €; is a pure random (white) noise, then we can estimate the coefficient g,, to 
minimize 
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2 


N (o) 
6D Gan - (6.3) 
n=] u=0 


However, €; is not necessarily white, and moreover it may sometimes have some 
feedback effect on the input. 


As already discussed in Section 2.6.3, even when noise exists in the output Y, of 
input X,, if there is no feedback effect as in Fig. 6.1, we can get rid of the effect of noise 
in computing the real spectrum of the output by finding the real response function of this 
system from the cross spectrum of the output to the input. 


‘LINEAR SYSTEM 


INPUT OUTPUT OBSERVED 


Fig. 6.1. Linear system without feedback. 


However, if somehow this output is fed back to the input, as in Fig. 6.2, the input X, 
is no longer uncorrelated with N,, and N; is included in X; as a part of the input. Accord- 
ingly, taking the cross correlation does not help in getting the real response characteristic 
of this system. 


LINEAR SYSTEM 


INPUT OBSERVED 


Fig. 6.2. Linear system with feedback. 


By a parametric method different from the nonparametric method referred to above, 
we can obtain the real relation by fitting an appropriate model to this system, getting rid 
of the effect of noise by following Akaike.>! 


Under these circumstances, we first transform the noise €; into pure white noise. 
Generally €; can be expressed by an AR model of arbitrary order /, 
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€; + Ci €44 + Cr2r€12 0 0 6 5R Cle 1 =€;, (6.4) 


€, being a pure random process. 


From Eq. 6.2 
lee) 
€; =Y, - yy Su Gens, (6.2’) 
u=0 
€n =Ypi- »y Ln Gein. (6.5) 
u=0 


Inserting these values into Eq. 6.4 gives, where 1] — 


Mig y, Bu Xrup + C149 Yr — » Su X1-u ¢ + C24 Yi-2 — y Bu X21 
u=0 u=0 u=0 


+...4+C) Gn SS ay ey shay rie = €;, (6.6) 
u=0 


or 
Y,+ CY) + C2Y;_2 tec: + Ci 1-1 Sy ten 
- > {au Xiu st Ci 8u Xiu + G3 8u X-2-u WP 610 6 
u=0 
+ C 8u Xiu + ne .} = €;, (6.7) 
Or 
YS Gp 4S > bu > Cr Xr + € 
1=0 u=0 =—«1=0 
= y Dena G; Xp + €;. (6.8) 
u=0 I=0 
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where (u’ = / + u) and u’ > /. Accordingly, if we set 


YS aes Ci = hy’, 
1=0 
Eq. 6.8 will be 
SS Gi ea =) lbp hn @ ey (6.9) 
1=0 u'=0 


Since €, is a pure random process, C; and h, can be estimated by the least squares 
method. 
From Eq. 6.9 


€,* = YY: Gyee »y [pips Lope (6.10) 
1=0 u'=0 


(* = conjugate) 
and 


ioe) io 2) 
Et+s = » C) Vis = i hy’ Xt+s—u'- (6.11) 
1=0 u'=0 


Taking the products of both sides of Eqs. 6.10 and 6.11, and then taking the statistical ex- 
pectation of each term 


R_(8)= > CG; > GC RyG)— », GY) hy. Ra(s—w' +) 
1=0 1=0 1=0 u'=0 


-> C1 >, hy Ry(s—1+u') 
1=0 =u’ =0 


+ > hy >, hy Rus). (6.12) 
u’=0 u'=0 


Then, by the Fourier transform, 
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Ryy(s) eda 
1=0 


1 SNS: am 
ae | Reet oii Sy Ci FC Seka | 


Sei oF > ba | Ray(s—u! +1") EHO dey 


-SC; en” yy h 
1=0 


A Wid 5 ' 
i eto Ry ens-u@ dw 
u'=0 


ioe) 


cf yo hi e +iu'@ > h,' eou'@ a | Rat ens” (6.13) 
u'=0 2m 
If we set 
NG en 1G; (@) ee Cre r() ee 
a. (6.14) 
Dem: = re (@yen > han eogen— Hi(@)), 
Eq. 6.13 becomes 
1 
Cen C1(@)C}()sy(@)— C7 @)Hy'(@)5xy() 
— C(w)Hy(w)syA@) 
+ Hy(w)Hy(@) - sy(@). (6.15) 
From this relation, we can get 5,,(@), Sy(@), ands, (sy,), and Eq. 6.15 can be written 
o2 * 
Sy(@) — Sy(@ 
=87 5 (Geayimen| ee Se (6.16) 
2m —Sy@) Sxx(w) | | Huo) 


Equation 6.16 shows the relation of the power of the white noise to the spectra of input 
and output and also of the cross spectra of input to output and the frequency response 
functions. 
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6.2.2. General Two-Dimensional, P—Dimensional Case 


More generally, as the expansion of Eq. 6.8, we can think of a system, diagrammed 
in Fig. 6.3, in which the output Xj, is fed back to the input X2, as well as the input X2, 


being fed to the output X) 


foe} le 2} 
Sena YS Chaim Konn & Epp 


Xt = 
l=1 u=1 
(6.17) 
lee} co 
269 = (Oriya SS, Cham Lone P CAN 
l=1 u=1 
Fig. 6.3. Two—dimensional feedback system. 
This equation can be written in vector form as 
Xie} _ | Cia Cra] | Xuer | | | C12 Crrz X12 
X21 Car Caza | | 420-1 C212 Cr2 | | 2.2 
Cc C Xi tk E 
+ Re agree eae rea | aa (6.18) 
Care Cr2¢ | | 220% €21 


Cutting off the terms at appropriate k by some criterion gives 
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X,= 4,X,_1+ a Xiot...+ 4X44 & 


k 
= y a, X4+ &, (6.19) 
u=1 
Xx Cc Ci €1 
ners x, m 1,t eS 11,m .2,m ee wt 
X24 C2.14m C22.m €24 
More generally, in the P—dimensional process the dimensions X),, X21, .. . Xps 


are connected by feedback systems to each other. This process can be reduced to the P— 
dimensional AR(k) model, with k as an appropriate order, 


X,= 4, X1+ 4, X9+...4+ 4 X 4+, (6.20) 
X] Chim 2 0-0 Cip.m €1 
X2 Com, O06 C2p,m €2 
where X, =| | an=|°  \ €= 
Xp Colm : Cpp,m Ep, 
Eg. 6.20 can then be written 
k 
X; = yy Gy, Xim+ Ey, (6.20) 
m=1 
or 
k 
1- > an(B™)} X= &, (6.20”) 
m=1 
or a(B) X,= €,, 
here 
ag)=|1-{4 Z+anZ+ + a,2']| 6.21 
= 1 2 0%O°0 k . ( G ) 
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Then the spectrum is 


Se (@ 
sx(@) = — (6.22) 
lace) 
2m - ay e”_ a> GMs. 6 =O GD 
where 

ERE Ne aa (6.23) 
€; €,] = ¢ 
yy t=0. 


The order k can be obtained from Akaike’s minimum AIC criterion in this case too; 
i.e., we can find k that minimizes 


AIC (p,k) = Nlog bu. | + 2p*k, (6.24) 
where 
k 
>) = > @mRx(m), (6.25) 
m=0 


as it was for the scalar case. 
Through these procedures, after fixing the order k, we get the coefficient 
a, ... Q,,as for the scalar case. Then we can get the spectrum matrix from Eq. 6.22. 


Srasa Sx 2 8 Sxix, 
Sxx; Sx 6 06 Sixx, 


: ; (6.26) 


The cross spectrum, for example S,,x,(w), is obtained in complex form 


Sx,x,(@ )= Cox,x,(@) a7 iOx 1%2(@) (6.27) 


as an element of the spectrum matrix, Eq. 6.26. 
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Then the frequency response function of x; to x2 is 


J {Coma} + {Ouy,2,(0)]? 


Gx,x,(@)| = 6.28 
Cx SG) (6.28) 
The phase relation is 
Qux,x,(@) 

x(@) = Arg[G,,x,(@)] = tan) 4 => — 6.29 
Px,x,(@) g1Gx,x,(@)] = tan Creal) (6.29) 

and the coherency function is 

ls.,2(@)? 
Vxj5 0) et (6.30) 
Sxix(@)Sxx,(@) 


6.3 AUTOREGRESSIVE CONTINUOUS PROCESS 


So far only discrete processes have been dealt with in which difference equations 
have been used to formulate the processes. When we study the response of some dynamic 
systems, however, the processes are in many cases substantially continuous. Usually, 
however, for computational analysis these processes are sampled at a certain time inter- 
val Ar, and then the readings are digitized and are treated as discrete processes. The 
analysis technique for a discrete process has been shown in detail in the preceding 
sections. 


Sometimes, however, it is helpful in understanding the response dynamics of a 
system to treat the process directly as the continuous process. 


Although difference equations have been used to formulate discrete processes, dif- 
ferential equations are used to formulate continuous processes. We are more accustomed 
to dealing with differential equations to express the physical characteristics of the re- 
sponse of the dynamic systems than with difference equations. 


In this section, the differential equations that formulate the continuous process will 
be introduced. Then, following the derivation of Pandit and Wu,* the relation of these 
differential equations to the difference equations which formulate the digitized processes 
will be summarized, since in practical applications, we might want to determine the char- 
acteristics of the differential equations from the digitized data of responses and inputs. 
63.1 The First Order Continuous Autoregressive Process A(1) 


For the first order continuous autoregressive process, referred to as A(1), the first 
order differential equation 


“x + @,X(t) = Z(t) (6.31) 


can be formulated. Here Z(t) is the forcing function or the white noise, expressed as 
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E|Z(t)| = 0 
E[Z(t)Z(t—u)] = 0(u)o% (6.32) 


= E[Z(t)P = 6(0)0z, 


6(0) is Dirac’s delta function, 


when u=0 
O(u) = 
0 when u ~ 0, 


and f end =e (6.33) 


Accordingly, E [Z(t)]* => ©, which means that the white noise Z(t) is physically 
unrealizable. The output X(z) is a stochastic process with zero mean. Using the differen- 
tial operator D = d/dt, D” = (d/drt)”, 

X(t) = (D + ao)}Z(t). (6.34) 


When we express Green’s function as G(V), 
[-<) t 


X(t) = | G(v)- Z(t—v)dy = | G(t—v)Z(v)av. (6.35) 


0 —0o 


This equation is the orthogonal decomposition of X(f), since the Z(t)’s are uncorre- 
lated, or independent at different times. From Eq. 6.34 and Eq. 6.35, it is also clear that, 
when the forcing function is Dirac’s delta function, X(t) is the Green’s function, 


(D + ao)G(t) = d(¢) 


G(t) = | G(v)d(t—v)dv. (6.36) 
0 
Here, since the solution of the homogeneous equation 
(D + ao) 'X() = 0 (6.37) 
is 
OSE CG, (6.38) 
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where A is the root of the characteristic equation 
A+Qo= 0, or A =-@Qo, 


X(6) —1Crenaes 


(6.39) 


(6.40) 


This coefficient C is determined from the initial conditions given by the characteris- 
tics of Dirac’s delta function as G(t) = 0, t< 0, G(0) = 1; therefore C = 1. Green’s 


function is obtained as 
Cae’ nia) 
G(t) = 
@ t<0 


Se) ete" 


here u(t) is the unit step function. 


The autocorrelation function is R(s) = E[X(t) X(t—s)]. 


Using X(t) = [ow Z(t—v') av’ 
0 


and X(t—s) = | oo Z(t—s—v) dv 
0 


gives 
R(s) ae | | Gv" GW )E[Z(t — v")Z(t— s—v) ]avdy' 
00 
00 
lo) cw s 
= 0% | G(v)G(v + s)dv = 0% | eA pAdlvts)p, — = eras 
0 0 
RCs) = R(s), 
thus 


211 


0 


(6.41) 


(6.41”) 


(6.42) 


2 
R(s) = SZ 6a Isl’ (6.43) 
2a0 


Now suppose we have the sampled discrete data from a continuous process with 
sampling interval Ar, and the data show an autoregressive process as shown by Eq. 5.7’ 


X,- 4X1 =€]. 


2 
a 

The covariance function is, from Eq. 5.49, R(r) = 7 <a’. R(r) = @2/1-a*)a’. 
=e 


Relation of A(1) to AR(1) by Covariance Equivalence. When s = rAt, the autoco- 


variance function for a continuous process must be equivalent to that of the discrete 
process at t = rAr. Setting s = rAt in Eq. 6.42 gives 


2 2 
R(rAt)= ar = =. (eeenie (6.44) 
0 0 


: 4 ate F C 
For discrete computation this is equivalent to R(r) = mee a’. 
@ 


Therefore 
eA! = g (6.45) 
2) 2 
(oj 
Ej (6.46) 
2a0 l-a 
or 
In 
—aoAt=I1na or Gees (6.45’) 
At 
and 
2 
o% = a (6.46’) 
—a 


From these relations, we can convert the continuous process A(1) into a sampled process 
AR(1), or vice versa, by inverting the coefficients from Eqs. 6.45’ and 6.46’. From Eq. 
6.43, the variance is 
2 
OZ 
RO) =—. (6.47) 
) ae 


The spectrum is given by the Fourier transform of R(s) as 


212 


s(w) = — | x. (oe ews ds 
220 


2 2 
1 
_ Ge, oer oe si ae (6.48) 
2x agtiwof 2 ap+@ 
Re {- } showing to take the real part of a function {-}. 


If we express 
(D+ ao) = aD), 


then 


s(@)=—4 —, (6.49) 


which is again a form similar to Eq. 5.50 for the discrete process. 


63.2 Correspondence of A(2) to ARMA(2.1) 
Generally for a damped mass spring linear vibrating system, the second order differ- 
ential equation as 


MX(t) + BX(t) + KX(t) = fit) (6.50) 


stands as its fundamental formulation, where f(r) represents the forcing function. 
Here 

M = mass 

B = damping coefficient (linear to the velocity) 

K = restoring coefficient (linear to the displacement). 

Transforming Eq. 6.50 by conventional expressions as 


B 
B Wag 28 
— = 2K0),, k= - — = 
M or 2 Gis Oe! (6.51) 
Ko re K 
Wie ns M 

gives 
x) +20 ! x) +a2X(1) = Re) (6.52) 
dt " dt Mi 
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Using the differential operator D = d/dr gives 


(D? + 2xaa,D + w2)X(t) = — ft, 
Or 


(D? +a,D + a)X(t) = — fit), 


where @; = 2KW,, Qo =>. 


Eq. 6.53’ is analogous to the AR model of the discrete process AR(2) as given by 


Eq. 5.59 or Eq. 5.62, 


X,+4)X1+@2X.2=€, or (1+a,;B+aB7)X,=€,. 


(6.53) 


(6.53’) 


If we assume (1/M)f(r) is a continuous pure random process Z(t), then we can say 


that X(r) is an autoregressive process of continuous time of order 2 and express this 


process as A(2) as did Pandit and Wu.*° 
Then A(2) can be expressed as 


(D? + 2kw,D + w2)X(t) = Z(t) 


o% when r=?’ 


where E[Z()Z(t')] = 03+ d(t-t') = 
0 when tr?’ 


which corresponds to Eq. 5.62 for AR (2), and 


X(t) = (D? + 2kw,D + w2)!Z(t) = (D? + aD + a) !Z(0). 


Then using the Green’s function G(t), 


X(t) = (D? + 2kw,D + w2)'Z(t) 
= | G(v)Z(t—v)dy, 
0 


and 
(D* + ayD + ao)G(t) = A(t) 
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(6.54) 


(6.55) 


(6.56) 


(6.57) 


G(t) = | G(v)d(t—v)av. (6.58) 
0 


Then thinking the characters of derivatives in Eq. 6.57, at r= 0, G’’(r) that is 
D*G(t) contains the same discontinuity as does 0(t). Therefore, G’(t) must contain the 
same discontinuity as does the unit step function, and similarly G(r), which is the integral 
of G'(t), behaves like the integral of the unit step function, which is a ramp function. 


Therefore, G(r) is continuous at r = 0, and the initial conditions are 


G'(th=G@a@=0, t<0 
G'(0) = 1, G(0) = 0. (6.59) 


From these initial conditions (Eq. 6.59), the coefficients C,, C2 of the solution of the 
homogeneous equation 


(D? + aD +a)G(t) = 0, (6.60) 
that is, of 
G(t) = Cy" + Coe", (6.61) 
are determined as 
1 =] 
Pe oe 6.62 
1 Tote 2 Tot (6.62) 
Therefore 
wv _ Art 
Be. for t>0 
Ay Az 
G(t) = (6.63) 
0 otherwise. 


Here A ,A2 are the eigenvalues or the roots of the characteristic equation equated to 
ZeTO, as 
fz) = Z* +a;Z+ ao = 0, (6.64) 


or 
Z* + 2kw,Z+ 0% = (Z—-A1)\(Z—A>) = 0. (6.64’) 


Then 
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Ay,A2 =5(-a: Se 63-420] 


or 
= a(— + a1). (6.65) 


This Green’s function is the so—called unit impulse response function that shows the 
response of this physical system to the unit impulse. 


ei ez 
ae) ios 


This impulse response function shows different forms as follows, depending on 


G(t) = (6.66) 


s 
AllV 
ee 


a. When k = 1 (a7 => 4ap), A,A2 are real. Therefore G(r) is a linear combination 


of two exponentials, as is shown in Fig. 6.4. 


Fig. 6.4. G(t) for «21. 


Then 
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KO pt 
= Fes sinh (0, \K- -—1 7) (6.67) 


KO pt i / 9 / 2 
G(t) = (<. a a (6.68) 
nvVKo— 


er@nt \ 
=—=> sin (0, Wiape 7) (6.68°) 


This equation is in the form of the impulse response function when x < 1, as in 
Fig. 6.5. 
c. When x = 0, G(t) shows an undamped sine curve as is also shown in Fig. 6.5. 


G(t) 


t exent 

K<1 
0 

—_—> | 
Git 
t xr=0 
0 

— > { 


Fig. 6.5. G(t)forx<1,x=0. 
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Using G(r) in Eq. 6.56 gives 


ew _ ow 

xX vt 

(t)= (eee eae ————_Z(t-v)dy 
0 


t 
1) _ Ar(t-v) 
= | eee ed Z(v)dv. (6.69) 


A, -A2 
—-o 
Equation 6.69 is the orthogonal decomposition of X(t) and also shows the general 
solution of the nonhomogeneous Eq. 6.54. 
The autocovariance function is then, by Eq. 6.56, 


R(s) = E[X(t) X(t-s)] 


= /3 [oonze—vrdv' | Gorze—s—vyd 
0 0 
| | G(v')GW)E[Z(t —v')Z(t—s —v)]dvdv' 
0 0 


| [ Gorrcose + s—v'av dv 
On0 


= 0% | G(v)G(v + s)dv. (6.70) 
0 


Inserting in Eq. 6.70 the expression G(v) = - 7 and manipulating gives 
1-42 


2 
B Ares Bee rate Tt EL 2. (671) 


O7 
SO Cee Ce) eee) 


Then the variance is 
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2 2) 
Oi Oo (6.72) 


0) : 
©) DAA, + A2) 4kw? 


The autocovariance coefficient 0(s) is 
Ages — Aye 


= 6.72’ 
COs cae (6.72) 


If this is a real process, then R(—s) = R(s) and 


5 isl Isl 5 
R(S\ = peu eae 2s) 6.71 
(s) Dalg(A2 aad | 2 1 ) ( ) 


Therefore, from the Fourier transform, the spectrum is, after manipulation, 


s(0) == | R(s)e5ds 


oF { Azet!s! 1 e42!s!y 
Qn 2 yA2(A2 —23) 


—-co 


e Sds 
0% 1 

2 
2x \iw? +a1(iw) +a 


eC ieee ee SS (6.73) 
2n (w*—w2) + 4k*w rw? 


If we use Eq. 6.54, 
(D? + Day + ao)X(t) = Z(t) 


a(D)X(t) = Z(t). 


Here a(Z) = Z*+a,D +o. 
Then from Eq. 6.73 
oO 1 ; 
(Oo (6.73’) 


This form is similar to Eq. 5.114, which is the expression for the discrete process 
AR(2). 
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Relation of A(2) to ARMA(2.1) by Covariance Equivalence. As was discussed in 
Section 5.3.5, ARMA(2.1) is more general and more flexible than AR(2). The Green’s 


function for AR(2) is composed of two exponentials as G; = emul + golly that is one step 


extension or complication of AR(1) process where the Green’s function is G; = pw that is 


one exponential. By the same token, we can say that ARMA(2.1) corresponds more 
closely or more generally and uniquely with A(2) than AR(2) does with A(2). 


Now let us express the ARMA(2.1) that corresponds with A(2) as Eq. 5.120 


X1+ aX; + A2X 1-2 = €, + DE 1-1, 
or as Eq. 5.123, 
(1+ a,B + a2B*)X, = (1 + bi B)e;. 


The autocovariance function for this ARMA(2.1) is from Eq. 5.166, 
R(r) = By + Boys, 


where /4;,//2 are the roots of the characteristic equation equated to zero for ARMA(2.1). 


Here 
fZ =Z? +a;Z+a2=0. 


From Eqs. 5.67 and 5.75 


My +2 =—-ay 


Mif2 = 42, 


(-«: =e ai —4a:) j 


Nile 


41,2 = 


and from Eqs. 5.168 and 5.169 


p, = 2 thy) | tbr woth 
(ui—M2)? | 1-wt 1 -wae 


_ Fein tbr) | wotbi mith 
(ui—M2)” | 1-ws 1—-pyr 


On the other hand, for A(2), when s = rAt, from Eq. 6.71, 


2 
OZ A rAt A rAt 
R At - 1 — 2 
ey) 2 A2(A4 73) G2 A ) 
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Therefore, if we put this in the form 
= dy} + dap, 
then 


07 


q, = ——_——_ 
: 24 1(Aj -A5) 


~o% 


a2 = —— 
2A —A3) 
Here /),A2 are the characteristic roots or eigenvalues of Eq. 6.64. Then 


Ay, +2 =-Q| 


A A2 = Qo. 


Then, as we did in Section 6.3.1, from the rule of covariance equivalence 


In 
aA = yy therefore a =A, 


In 
At = > therefore a2 =A, 


and also 
dts Ah! = y+ =-aQ\ 
eth = oith2)Ar — Miu2 = a2 ‘therefore Inaz = (Ay +A2)At =—ayAt 
In a2 
a= f 
Ie Ae 


Equating the values of d,d2 for A(2) as in Eq. 6.75 and By, B2 for ARMA(2.1), 
expressed by Eqs. 5.168 and 5.169, gives 
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(6.74) 


(6.75) 


(6.76) 


(6.77) 


(6.78) 


(6.79) 


(6.80) 


(6.81) 


Of _ Oe thi) | itd: wots 


WiAji—-A3) i-me)? | 1-we 1 -pye 


(6.82) 
Gr Ge) | ae Phe 
2A2Aj—AS) 1-2)” | 1-n3 1- mye 
Taking the ratio on both sides of the two equations in Eq. 6.82 gives 
ry Mtb) woth 
bi) et 2 re eare 
GAs 2 NE ore ae (6.83) 
a b b 
+b Hot0; Mith 
(U2 + by) rae ei 
This is a quadratic equation in b, and has the form 
b3+2Pb,+1=0. (6.84) 
Thus 2P =-| by Mes s (6.85) 
b; 
After manipulation, we get 
op = Ai +e =H») SG LIES) (6.86) 
Ay — 13) —Anu2(1 — 7) 
and also 
bj =-P + VP*-1. (6.87) 


The homogeneous equations of Eqs. 6.53 or 6.54 is 


(D? + aD + ao)X(t) = 0 
(D? + 2k, + w2)X(t) = 0, 


and its eigenvalues A;,A2 are, according to Eq. 6.65, 
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1 
ize 5(-a =e J 4a) =—a,(« + Pai) 


We can transform Eq. 6.86 into the equation expressed by @,a@2, and b; of the corre- 
sponding ARMA(2.1) model by the relation of the so—called covariance equivalence. 


a. When at <4ay (k*<1): 


Further, for simplicity of expression, we set 


A, =-—utiv 
(6.88) 
An = —u-iv. 
Then 
CO eee el 
NA a 4a9- a] = >On Lore 
6.89 
1 (6.89) 
u=—Q\ 
Q,=2u 
(6.90) 
Qo = u? + v?. 


From the covariance equivalence relations between AR(2) and ARMA(2.1) we have 
Eqs. 6.77, 6.78, 6.79, and 6.80. Further use of Eq. 6.88 gives 


— Ay = fy + fz = EMAL 4g. HWIAT = oul. 2 cos(vAt) 
fy — M2 = eo“ 23 sin (vAt) 
1 —az = 1—pyn = eo“ -2 sinh (uA) 
L+ap=14+pyz =e’ -2 cosh (uAt) Cc) 
pi — a = 2e“' .i sin (2vAt) 
1 —a3 = 1 —ptud = 2e°*™ - sinh (2uAz). 


Inserting these expressions into Eq. 6.86 gives 
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~ 2u sin (vAt) cosh (uAt)—2v sinh (uAt) cos (vAt) 


Then, 
u [2P sin (vAt) cosh (uAt)+ sin (2vAr)] (6.93) 
v= ooo. 5 
2P sinh (uAr) cos (vAt)+ sinh (2uAr) 


From Egs. 6.90 and 6.81 


we eA — [ay (6.94) 


Therefore, from Eqs. 6.91 and 6.94 


2 Vaz 
\— (ai — 4a) 


sin (vAr) = Ales 
2 


cos (vAf) = 


(6.95) 


1+ 
cosh (uAr) = 22 


1-ap 


2Vay 


sinh (uAr) = 


With Eqs. 6.95 and 6.85, Eq. 6.93 can be transformed into an equation with a), a2, and 5), 


— ik —2a,;+(1+a)(b) ++ 
we Sa) V— (a3 — 4a) 22+ Ctan(ritd) (6.96) 
2Ar 2(1 — a3) —ay(1 —a)(b, +2) 
If we look at the first equation of Eq. 6.95, v is derived as 
14 [ee (6.97) 


1 
vV=— cos” : 
At 2 Va, 


That value is not unique, since it does not include the MA parameter b,. However, 

Eq. 6.96 shows that v is uniquely determined from aj, a2, and by, since it includes the 
MA parameter b, as well as a; and a2. This is another reason why generally we prefer 
ARMA(2.1) over AR(2) as the discrete process corresponding to A(2) and u is expressed 


by Eq. 6.91. 
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After u, v are determined, @},Qo, or R,w,, can be easily derived from u, v by 
Eq. 6.90. 


b. When aj = 4ao (k?>1): 
We can set 


Ay =-utv 


6.98 
A2 =-Uu-V. 


Then computing the necessary elements for P in Eq. 6.86 in a similar way, we get 


My t+h2= iAt 4 pA2At — euAt. 2 cosh (vAr) 
fy —M2 = eo“! .2 sinh (vA) 


1 —pypz = eA’ .2 sinh (uAr) 

(6.99) 
1 + Myly = e““" 2 cosh (uAt) 

(41 +2) (Ur —f2) = 2 € #4" sinh (2vAr) 
(1 —p2u3) = 2 &*' sinh (QuAd). 


Inserting these values in Eq. 6.86 for P gives, after manipulation, 


u sinh (2vAr)—v sin (2uAr) 


= (6.100) 
2u sinh (vAt) cosh (uAt)—2v cosh (vAt) sinh (uAt) 


Then 


be u [2P sinh (vAt) cosh (uAt)— sinh (2vAr)] (6.101) 
2P cosh (vAt) sinh (uAt)— sinh (2uAr) 
As were for at < 4a, the sin, cos, sinh, cosh functions are expressed by a, a for 
this case. Therefore, after v and u have been determined, @1,@2 or K, w,, can be deter- 
mined from u, v easily. Their derivations are omitted here because we are usually less 
concerned with the case when k*>1. 
For this case, the coefficients @),Q@o or K, w, of the differential equation are 


derived from a, a2, and b; uniquely. The differential equation which expresses the con- 
tinuous autoregressive process can be obtained from the parameters a), a2, and b, of the 
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difference equation that formulates the discrete autoregressive and moving average 
process ARMA(2.1). 


The above mentioned is for the simple case of ARMA(2.1) versus AR(2). For the 
more complicated cases of ARMA(4.3), ARMA(6.5) ..., theoretically A(4), 
A(6),..., Should correspond, and the parameters of differential equations that formu- 
late these A(4), A(6)... should be derived from the parameters of the difference 
equations that formulate ARMA(4.3), ARMA(6.5) ..., although their relations might 
be much more complicated than for A(2) from ARMA(2.1). The differential equations 
that formulate the continuous autoregressive process give us good clues to finding the 
physical characteristics of their response processes. 


The second order linear differential equation is the basic equation of a linear dynam- 
ic system with one degree of freedom. Accordingly, the above discussion indicates that 
the ARMA(2.1) process represents the dynamic behavior of a linear system with one 
degree of freedom under the excitement of white noise. 
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CHAPTER 7 


EXAMPLES OF MODEL FITTING TECHNIQUE APPLIED TO THE 
ANALYSIS OF SEAKEEPING DATA 


In this chapter, examples of application of the model fitting technique to seakeeping 
data will be presented to demonstrate the applicability of this technique to the analysis of 
these data. 


7.1 EXAMPLES OF AR(n) MODEL FITTING FOR THE PREDICTION 
OF SEAKEEPING DATA 


Figure 7.14° shows examples of fitting of AR(n) models to observed seakeeping 
data. After the appropriate order number n was determined by the MAIC method, as 
described in Section 5.5.4, the parameters a ... a, were calculated from 800 observed 
data points for the respective processes. Here for rolling n=7 and for swaying n=19 were 
found to be the optimum. Figure 7.1 shows the values predicted for each process by 


Xp = — 4 Xj-] — A2K_-2 —- —— ApXi-n (7.1) 


PREDICTION START 1 - STEP AHEAD FOR ROLLING 


a 
a 


DATA LEVEL 


} 
0.00 1050 21.00 |3150/ 4200 S250 63.00 7350 84.00 4.50 105.00 115.50 126.00 136.50 147.00 157.50 168.00 178.50 189.00 199.50 210.00 
~a.04 Se 7 

TIME ~ 


PREDICTION START 10 - STEP AHEAD FOR SWAYING 


DATA LEVEL 
‘ 2 
8 
Se 


‘0.00 1050 21.00 3150 ,4200 5250} 63.00 A 94.50 105.00' 115.50 126.00 413650 147.00, 
~ TIME \ 


PREDICTION START 1 - STEP AHEAD FOR SWAYING 


} t 
0.00 1050 21.00 31.50 p4200 S250} 63.00 73.50 | 64.00 }94.50 105.00_ 115.50 126.00 £136.50 147.00 
\ Si if \ 7\ TIME Vy! V 


DATA LEVEL 


157.50, 168.00 178.50 189.00 199.50 210.00 
y 


\ 


Fig. 7.1. Comparison between the predicted values of seakeeping data 
and the observed values. 


(From Yamanouchi, et al.46) 
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using these a; . . . a,. One—step—ahead predictions for rolling and swaying and a 
10— steps—ahead prediction for swaying are shown here. 


7.2 EXAMPLES OF MODEL FITTING TECHNIQUE APPLIED 
TO THE SEAKEEPING DATA 


Figure 7.2*° shows examples of spectra of model seakeeping data obtained in a 
model basin for a model of a ship. On the left-hand side (a), the spectra calculated by the 
nonparametric method, the so—called Blackman—Tukey method, are shown and on the 
right-hand side (b), the same spectra calculated by the parametric method, i.e., the AR(n) 
model fitting technique. The spectra for wave height, heaving, and relative wave height 
are shown on a logarithmic scale, and the Nyquist frequency of 2.50 shows 
At = (1/2.50 X 2) = 0.2 sec. The total number of observations is N = 225, which is 
rather small, and the maximum lag number for the B—T method is 50, which is large com- 
pared to N. We can find the spectrum from the AR model, the order of each process being 
shown by the AR number in each figure. The spectra are very smooth and the peaks of 
the spectra are sharply defined even from these short records (small number of observa- 
tions). 

Figure 7.3% shows the behavior of AIC values that were used to find the order n of 
the AR models fitted to each set of these seakeeping data. From this figure, we found n to 
be 11, 9, and 10 for wave height, heave, and relative wave height, respectively. These 
values give the minimum AIC. 


Figure 7.4 shows the time series of an AR(2) model simulated by the difference 
equation, 


X- 0.5X,-1 + 0.7X,_2 = Ey 


when a, =—0.5, @2 = 0.7 in Eq. 5.59 for a general AR(2) model. There, €, is white noise 
with Gaussian distribution N(0, 1), i.e., with 0 mean and a variance 1. The time series are 
also shown in the same figure. The number of observations is N=1,000 for both time 
series €, and X;,. The theoretical spectrum of this AR(2) model is shown at the top of Fig. 
7.5.52 An AR(n) model was fitted to the simulated process, and from AIC criteria, the 


optimum n was estimated as n=2. The coefficients a), a2, and the variance ad of €; were 
estimated by the method described in Section 5.2.3, and the spectrum s(q@) was calculated 
using these parameters, as shown at the bottom of Fig. 7.5. Independent of this order n=2 
by minimum AIC criteria, spectra were also estimated for higher n; A(10) and AR(20) 
models were fitted and their spectra were obtained as shown in the same Fig. 7.5. 
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Fig. 7.2. Examples of spectra of model seakeeping data. 
(From Yamanouchi, et al.46) 
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Fig. 7.3. Behavior of AIC for seakeeping data in getting the spectra (b) in Fig. 7.2. 
(From Yamanouchi, et al.4°) 
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Fig. 7.4. Simulated AR(2) process X, = 0.5 X,, — 0.7 X,2 + &, 
€; is a white noise N(0,1). 
(From Yamanouchi, et al.52) 
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(a) AR method (optimum). (b) AR method (not optimum). (c) B—T method. 


Fig. 7.5 Comparison of AR-model fitting method and B—T method and the effect of 
changing orders of AR-model fitted to the simulated AR-2 process. 


(From Yamanouchi, et al.52) 
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For comparison, the spectra calculated by the nonparametric method (Blackman— 
Tukey method) are also shown in this figure. The maximum lag numbers are 50 and 100, 
which give a fairly high confident estimate since these numbers are 1/20 and 1/10 of the 
total number of observations N=1,000. It is interesting to note that the result for lag=50 
looks much like the one for AR(20). The impressive point here is, of course, that the esti- 
mate from the model fitting technique, which determines the order from the minimum 
AIC criteria, gives a result that checks the theoretical process very well. 


Figure 7.6 compares the estimates from AR model fitting and the B—T method for 
the same simulated process (Fig. 7.4) for a wider range of order (AR 1—16) and maximum 
lag numbers of 20-500, for reference. 


Figure 7.7°* shows a few examples of the analysis of actual at—sea ship performance 
data. Rolling, wave height measured by buoy, and the horizontal acceleration of the en- 
gine bed (as of different frequency characteristics, because the engine bed is mostly 
excited by the prime movers of the ship) were analyzed by three different methods: the 
AR model fitting method, the F.F.T. method, and the Blackman—Tukey method. The sam- 
pling time interval was Ar=1 sec for roll and wave height, Ar=0.02 sec, much smaller for 
acceleration. The spectrum of the horizontal acceleration of the engine bed estimated by 
the AR model fitting method gives very reasonable results, compared with the ones ob- 
tained by other methods, the former clearly showing the existence of multiple natural 
frequencies. 


Figure 7.8>° shows other examples of the spectra of the horizontal acceleration of 
the engine bed estimated by the AR model fitting method at orders n of 5, 10, 15, 20, 30, 
and 55. The result at n=10, where the AIC shows minimum values, looks most reasonable 
and clearly shows the existence of multiple natural frequencies in horizontal vibration. 


7.3. EXAMPLES OF PARAMETRIC ANALYSIS OF RESPONSE 
CHARACTERS OF MARINE VEHICLES AND STRUCTURES 


Figure 7.9°? shows an example of simple response, the case of yaw angle versus 
rudder angle of a seagoing ship. The results for the spectra of input and output and the 
frequency responses from two methods, the AR model fitting and nonparametric meth- 
ods, are compared. When the spectrum and response characteristics do not show any 
abrupt changes with frequency, as in the example, most of the results from the two meth- 
ods look similar. However, when we look at the Nyquist diagram or the mode diagram of 
the frequency response characteristics, the results from the AR model fitting are much 
better and smoother and more reasonable, physically, than those from the nonparametric 
method. 


Figures 7.10 and 7.11*° show the results of seakeeping data for a model ship in 
waves produced in the towing tank, also analyzed by two different methods. 


From these figures we see that the AR—model fitting techniques give smoother 
curves for the spectra and for the frequency responses. Also, in the important range of 
frequencies, the coherency values are closer to 1, showing that the response characteris- 
tics are more reliable when estimated by this method. 


In both figures, the cross spectra are shown by their real and imaginary parts, i.e., 
by co— and quadrature—spectra. The shape of these spectra, which show very sharp peaks, 
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2.0 X,=0.5X,_4-0.7X_p +e 
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2.0 ag Ono! molshatolaiumols 
AR-2 L=50 
° 0 


2.0 AR-4 L=100 
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2.0 AR-8 
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20 AR-16 
0.1 0.2 03 0.4 0.5 0.1 #02 03 04 0.5 
—_ w —_ w 
(a) AR method. (b) B—T method. 


Fig. 7.6. Effect of changing orders in AR—model fitting to the simulated AR(2) 
process (Fig. 7.4) [MAIC shows order is AR—2] and the comparison 
with spectrum obtained by B-T method. 


(From Yamanouchi, et al.5?) 
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(a) AR model fitting. (b) FET. method. 
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Fig. 7.7. Comparison of the spectrum through AR—model fitting and 


conventional methods. 
(From Yamanouchi, et al.52) 
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Fig. 7.8. Effect of changing the orders in AR—model fitting (by MAIC; AR=10). 
(From Oda, Yamanouchi, et al.°°) 


indicates that the AR method gives better results than the B—T method and more precisely 
follows the rapid changes in the curves. 


By the way, the time shift techniques that this author proposed in Section 3.2 might 
have given even greater improvement in the results if it were properly applied. The rapid 
change in phase response with frequency shows this fact and indicates that there is still 
room for improvement by the proper shift of responses even in the case of using model 
fitting techniques. Figure 7.12** is an example of the same kind of comparison of two 
analysis methods used on the pitch motion of a model of an offshore semisubmersible as 
shown by Fig. 7.13 on irregular waves in the model tank. 
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Fig. 7.10. Comparison of methods to analyze the frequency response characters 
of a model ship in the tank — sample 1. 


(From Yamanouchi, et al.4°) 
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(b) AR—model fitting method (order n=14). 


Fig. 7.11. Comparison of methods to analyze the frequency response characters 
of a model ship in the tank — sample 2. 


(From Yamanouchi, et al.4°) 
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Fig. 7.13. Photograph of a semisubmersible model (scale 1/90). 
(From Oda, Yamanouchi, et al.5%) 


Figures 7.14~7.17>° show the results of frequency response analysis of the model 
seakeeping data as heave, relative wave height, pitch, and surge in irregular waves deter- 
mined by the AR model fitting technique for the same offshore semisubmersible. These 
data were analyzed as a one—input/one—output vector process, although, if we like, we 
can analyze them as a one—input/multiple—output vector AR process. The orders of AR 
models fitted to these systems are marked on each of the figures. 


Figure 7.16 is concerned with the same response under the same conditions as Fig. 
7.12. From these results, we can find the same characteristics of AR—model fitting tech- 
niques. All the spectra and the frequency responses are much smoother and follow well 
the rapid changes with the frequency. These tendencies are more clearly indicated by Fig. 
TUS: 


In Fig. 7.18 the response characteristics of heave, relative wave height, pitch, and 
surge are shown expanded in the important range of response frequencies for 6-24 sec- 
ond periods for a full—scale structure. The results obtained by the AR model fitting and 
B-T methods are shown with results of experiments in regular waves in the tank. The 
figures show rather surprisingly how well the results from the AR model fitting technique 
follow the minor changes of the response in amplitude as well as in phase relations ob- 
tained in experiments with the regular harmonic waves of several frequencies. 
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Fig. 7.14. Frequency response analysis for heave by AR—model fitting 


(N=250, At=0.2 sec, AR-8). 


(From Oda, Yamanouchi, et al.59) 
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Fig. 7.15. Frequency response analysis for relative wave height by 
AR-—model fitting (N=250, Af=0.2 sec, AR-15). 


(From Oda, Yamanouchi, et al.53) 
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Fig. 7.16. Frequency response analysis for pitch by AR-model 
fitting (N=250, At=0.2 sec, AR-8). 


(From Oda, Yamanouchi, et al.°?) 
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Fig. 7.17. Frequency response analysis for surge by AR—model 
fitting (N=250, At=0.2 sec, AR—11). 


(From Oda, Yamanouchi, et al.5°) 
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Fig. 7.18. Frequency response characters by different methods and 
through experiments in regular waves. 


(From Oda, Yamanouchi, et al.59) 
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CHAPTER 8 


CONCLUSION FOR PART I 


In Part II, we have discussed the model fitting technique or parametric analysis 


method for a random process. Comparing this method with those discussed in Part I for 
nonparametric analysis, we can conclude as follows: 


1 


tv 


The examples in Sections 5.2.1 to 5.2.5 show that the autoregressive (AR), mov- 
ing average (MA), or autoregressive moving average (ARMA) model of rather 
low order can represent pretty complicated looking processes with various 
frequency characteristics. 


For a given process, generally we can fit an autoregressive (AR), moving average 
(MA), or autoregressive moving average mixture type (ARMA) model of certain 
finite order. From the sample autocovariance functions at rather low lag num- 
bers, we can estimate the finite number of parameters of the model and the 
variance of the residual errors, once the order of the model has been determined. 


The characteristics of the discrete parameter model are closely related to those of 
the correlations of that model. To find the finite number of parameters to express 
the process, the correlation function until the finite order is sufficient, as men- 
tioned in item (2). Sample correlations at low lag numbers are more reliable than 
those at higher lag numbers, and there is no need to be concerned about the lag 
windows as there is in the nonparametric analysis (the so—called Blackman— 
Tukey method), where, ideally, we need the correlation function over 


— © to + © of the lag numbers. This concern about lag windows and consistent 
estimates, resulted in many efforts to find good windows. Not using the lag win- 
dow is also one reason why we can get better results by the model fitting 
technique then from rather short records. 


The order of the AR(k), MA(J), or ARMA(k, J) models that are to be fitted to a 
sample process, as the best from a statistical point of view, can be determined by 
Akaike’s information criteria (AIC) as the combination of the parameters (k, /) 
which minimizes AIC. 


Evaluation of the parameters of the AR(k) model is easily performed by solving 
the Yule Walker equation, which is a linear relation of parameters. The solution 
by Yule Walker’s equation is a good approximation of the solution by the maxi- 
mum likelihood and least squares method. On the contrary, for the MA and 
ARMA models, estimation of parameters is more troublesome and difficult and 
involves solving the nonlinear least squares equations. 


As far as stability and invertibility are fulfilled, the AR and ARMA processes of 
finite order can be inverted to the MA process of infinite order and, conversely, 
the MA and ARMA processes of finite order can be inverted to an AR model of 
infinite order. Usually, however, in this approximation, the order increases from 
the original MA or ARMA order. The order of the AR model that best approxi- 
mates the original MA or ARMA is obtained through the AIC criteria. In this 
approximation, estimation of parameters becomes much easier than for the origi- 
nal MA or ARMA process as mentioned in item (5). Accordingly, AR model 
fitting is usually used practically for general processes. 
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16. 


After an appropriate AR, MA, or ARMA model has been fitted, it is easy to cal- 
culate its spectrum. The spectrum function can be estimated analytically from 
only a few parameters that characterize the fitted model. The spectrum thus ob- 
tained is smooth compared to the spectrum computed by the nonparametric 
method, or by the Fourier transform of the estimated autocorrelation. A sharp— 
peaked spectrum can be obtained by this method, free from the smoothing or 
blurring effects of the spectrum window that are inevitably applied in the 
nonparametric method. 


From the characteristics of the model fitting technique, we can get a fairly reli- 
able and sharp spectrum even from rather short records of observation, when 
reliable autocorrelation functions at large lag numbers are difficult to esumate. 


The AR model is effective in finding the peak frequency and in estimating the 
peak value of the spectrum. Usually the second order of an autoregressive model 
is necessary to identify one peak of the spectrum. 


When the purpose of the analysis is to estimate the characteristics of responses, 
the AR model that estimates the peak effectively is an appropriate approximation 
of the ARMA model. The ARMA model, however, is more reasonable for ex- 
pressing the response of a linear dynamic process to random excitations, as will 
be mentioned later. 


To express the flat valley of a spectrum, however, a large order becomes neces- 
sary if an AR model is to be fitted. A flat valley is more easily expressed by an 
MA model of low order. This tendency comes from the character of the trans- 
form function of the spectrum. 


The ARMA model that expresses peaks by an AR model and flat valleys by an 
MA will be the most appropriate in general cases. 


The model fitting technique relates all linear stationary processes to appropriate 
AR, MA, or ARMA models. All of these models assume that the process is the 
output of a pure random process or of white noise. The linear relations of this 
process to white noise can be derived as the Green’s function of these models. 


When a process is the linear output of some input that is not necessarily white 
but colored, we can get the response of this process to the real white noise if we 
assume that the colored input is the output of the real white noise. In this way we 
can relate all the colored input to a random process. 


Even when the linear response system has some feedback effects, we can fit a 
vector autoregressive process, inverting the input into a pure random process by 
applying an autoregressive process technique to the input, as mentioned in item 
(14). We can get the linear response characteristics of the system from the 
elements of the spectrum matrix of the vector process. In observations of sea- 
keeping data, sometimes the feedback effect is concealed, so this method can be 
applied effectively in the analysis of such data. With the conventional nonpara- 
metric method, the kind of system that has feedback effects is hard to handle. 


A second order autoregressive continuous process A(2), formulated by a second 
order linear differential equation of the damped mass spring vibration type, can 
be transformed into a discrete ARMA(2.1) process formulated by a difference 
equation and by the equivalent correlation theory, the coefficient of the 
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20. 


differential equation can be expressed by the coefficient of difference equation 
and visa versa. The second order autoregressive continuous process A(2) is then 
the basic process, as the response of a linear vibration system to the white noise 
with one degree of freedom. Conversely, when a dynamic process with one 
degree of freedom is excited by white noise, the digitized response can be ex- 
pressed by an ARMA(2.1) model. When the input is colored and not white, we 
can invert this system to a pure random process (white noise as mentioned in item 
14) although the order increases. Thus the ARMA model is the most general pro- 
cess for expressing the oscillatory response of a linear system. 


An advantage of the parametric analysis is that, by fitting a certain discrete model 
to the response process, theoretically we can estimate the characteristics of the 
differential equation that governs the response of a continuous process by invert- 
ing the difference equation that expresses the discrete model into the equivalent 
differential equation that formulates the continuous process. 


The confidence limit of each parameter of a fitted discrete model can be eva- 
luated. The remaining statistical considerations are taken care of by adopting the 
most adequate order of the model based on Akaike’s information criteria. How- 
ever, if we could express the overall reliability by some simple expression, say, 
for example by a confidence band at some level of probability, it would be easier 
to give us more confidence in the parametric method, directly comparing with the 
result by the nonparametric method: 


The AR model was adopted here first and was applied to the seakeeping data of 
ships and offshore structures, both for models in the tank and for the actual ships 
and structures at sea. From the results, we found this model fitting technique was 
practical to apply and was a promising method of analysis, sometimes giving us 
better results than the conventional nonparametric method. At least this model 
fitting or parametric analysis technique supplements the nonparametric method. 


The ARMA model is the most general and is directly connected with the continu- 
ous AR model or its formulating differential equations that govern the behavior 
of the system. It seems to be the most desirable for estimating the physical char- 
acteristics of the process, although its handling in the estimation of parameters is 
more complicated than for a pure AR model. Accumulation of experience with 
this application is now very much needed, this author believes. 
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APPENDIX Al 
DATA FOR THE GENERATION OF THE PROCESSES 
Data for the generation of a pure random process AR(1), AR(2), ARMA(2.1), 


MA(2), MA(1), and ARMA(2.2), given in Chapter 5, Figs. 5.3, 5.10, 5.20, 5.25, 5.28, 
5.32, 5.35, 5.38, and 5.41, and Tables 5.1 and 5.2 are listed on the following pages. 
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APPENDIX A2 


POLYNOMIAL MODEL FITTING TO OBSERVED DATA*® 


Given pairs of data {X;, ¥;},i=1- - -N, we often want to express Y; as a 
polynomial of X;. 
If we fit yy.;, a polynomial of x; of order M, as 
Pei = Ag+ aX; + nx ++ > - + aye (A2.1) 
p= 1, 
we can estimate the coefficients dp, a,- : - d,, by the least squares method. Namely, 
computing the mean square of the difference 
N 
1 A 
Ou= W > 0% —Jmiy, (A2.2) 


i=1 


and find the ao, a) - - dm that makeo, minimum. Then from 


80 9 ay+(> xi)a1 +: +(S au = > yi 


0ao 


(A2.3) 


it «0 (Sixi)ao+ (Sx? )ai + : + (Sal ay = ic 


80 (Sx a9 + (Satay +: + (S32 \ay = ye. | 


day 


The do: - - ay are calculated as the solution of the simultaneous Eq. A2.3. 
Here, the most serious problem is determining the order. We can make yy; as close 
as we wish to y; by making the order M very large. However, if M is too large, yi 


follows, even to the random error of the observations. Of course, if M is too small, yy; 


sometimes neglects the variations in the data that really exist, as shown in Fig. A2.1. 
Order 1 is too small, but order 5 seems too large. 
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ORDER 2 


Y-AXIS 


0.00 0.80 1.59 2.39 3.18 3.98 4.78 5.57 


X-AXIS 


Fig. A2.1. Fitting of polynomials to the observed data. 
(From Yamanouchi, et al.4®) 


All observed data include the statistical error. Accordingly, when x; are given, 
instead of Eq. A2.1, y; should be represented by a regressive polynomial, 


Vi = Ag + Q4X; + ax? +- ° -+ayx +e;. (A2.4) 


Here €; is a probability variable that follows the normal distribution with 0 mean and 


variance a2. Here, we can define that model fitting is the statistical selection of a model 


to fit best to data in statistical meaning. When we use Eq. A2.4 as the model, we can use 
the minimum AIC criteria or the MAIC method to determine the order M. When y; is 


approximated by a polynomial of order M as in Eq. A2.1, the coefficient ;’s has the 
values determined by the least squares method, 


N 
—L=min(1/N) >) Oi-yy,))- (A2.5) 
i=] 


Then let us investigate the behavior of this mean square of the residual error 


1< Yo 29) 
Wo ae =¢ 


1=1 


or N times the logarithm of Eq. A2.5 
Nlog ) Qi-Sy, = Nloga?. (A2.6) 


This value A2.6 is shown in Fig. A2.2 by the mark O and tells us that the larger the value 
of M, the smaller this value A2.6 is, or the larger the order M, the smaller the residual 


9) 
error G¢. 
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UD 


NO ww 


-25 i 


-30F 


AIC Nx LOGu? 
a) 
ol 
a 


NO. OF PARAMETER (M + 1) 


Fig. A2.2. Behavior of AIC for Fig. A2.1. 
(From Yamanouchi, et al.4®) 


However, this procedure of guiding M from Eq. A2.6 does not give us the best 
values of the order, because Eq. A2.6 does not include the goodness of fit to the real 
structure of the data or the decrease in reliability with the increase in the number of 
parameters to be estimated. So, to show the penalty for this increase in the order, if we 
add the term 


2 X (no. of parameters) = 2(M + 2) (A2.7) 


to Eq. A2.6, that gives us the AIC, 


N 
AIC(M) = Nlog >’ (vi-Jm,i)” + 2(M + 2). (A2.8) 


t=1 


The number of parameters M + 2 comes from (M+1) 4;'s for do, a1: - - ay plus 1 
for o * that we must also estimate. The behavior of 2(M + 2) is shown in Fig. A2.2 by 
the mark 0 as a straight curve that increases linearly by M. Then AIC, expressed as the 


sum of N logo * and 2 (M + 2), behaves like the 4 in Fig. A2.2 and shows a minimum at 
a certain M, here M + 1 = 3 for the data shown in Fig. A2.1. Thus the order of the 
polynomial is determined as M = 2 by AIC criteria. 
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From the discussions in Section 5.4.3, we know that Eq. A2.5, the estimation that 


N 
minimizes (1/N) S60: —yu, es is the estimate that maximizes the likelihood function L, 
i=] 


or the logarithmic likelihood function log L. Accordingly, Eq. A2.8 can be written as 
AIC = — 2log(max likelihood) + 2(no. of parameters). (A2.9) 


Eq. A2.9 supplements the characteristics of AIC, this author believes. As another 
example of polynomial fitting, Fig. A2.3 shows the plot of the observed data for a 
maneuverability test of a ship. 


By the same procedure, from the behavior of AIC in Fig. A2.4, the order was 
determined as M = 5, and the fifth order polynomial was estimated as the best fit to these 
data. 


9 
a 


id 
> 
STARBOARD 


0.2 


YAWING RATE (DEG/SEC) 
° 
w 


HELM ANGLE (DEG) 
10 20 30 40 


STARBOARD 


ORDER 5 
ORDER 8 


Fig. A2.3. Fitting of polynomials to the data of maneuvering test of ship. 
(From Yamanouchi, et al.4°) 
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On Fig. A2.4, the behavior of AIC is shown for this fitting. 


-210.0 min. 


-220.0 y 


-230.0 I cn 
0 1 2 3 4 5 6 U 8 9 10 


NO. OF PARAMETER (M + 1) 


Fig. A2.4. Behavior of AIC for Fig. A2.3. 
(From Yamanouchi, et al.4®) 
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PART I 


TREATMENT OF NONLINEARITIES 


CHAPTER 9 
INTRODUCTION 


9.1 INTRODUCTION FOR PART III 


In the preceding sections, Part I and Part II, the processes were assumed to be al- 
most linear, and in so far as the process is linear, the periodogram analysis shown in Part I 
and the model fitting method in Part Il are effective methods of analysis. For example, a 
Fourier—Stieltjes expression for the process assumed to exist in Part I is originally a linear 
expression, and the AR, MA, or ARMA models treated in Part II are based on the idea of 
decomposing the processes into independent or orthogonal processes, i.e., assuming their 
linearity. However, if the process is nonlinear, these methods can no longer be applied 
directly. We need some special considerations for their treatment. 


In most engineering fields, many phenomena can be approximated as linear. How- 
ever, no phenomena are purely linear but include some elements of nonlinearity. Today in 
nonlinear phenomena such as the effect of viscosity, the secondary potential forces that 
include the interaction of two frequency components of excitations have come to be con- 
sidered important in treating the behavior of ocean vehicles and structures, although 
useful information has been derived even under the limitations of linear approximation. 


Here in Part II, the nonlinearity of waves themselves that are the source of excita- 
tion to systems of ocean vehicles and structures is first investigated. According to a few 
works already published, their nonlinearity is usually not large. 


The approximation methods that have been used to treat nonlinear response pro- 
cesses, such as the linearization method and the perturbation method, are summarized. As 
more advanced approximations, the Voltera or functional expansion of the nonlinear pro- 
cess and the application of polyspectra are then introduced. 


As an extension of these approximation method, a slightly different aspect of the 
analysis of the probability characteristics of the process, the probability distribution func- 
tion of the extremes, is summarized. As extensive work has been published recently on 
the analysis of nonlinearity response including these probability characteristics by J. F. 
Daizell,!*!3.55-8 only the derivation of general characteristics of several functions and 
the results of these applications are reviewed. 


Next in Part III, the treatment of stochastic processes as Markov processes is intro- 
duced, and then the application of the Fokker—Planck equation, also recently introduced 
by J. B. Roberts>*! in the analysis of seakeeping data, is reviewed. 


Finally, as a slightly different approach, or as the extension of the model fitting tech- 
niques discussed in Part II, a few examples of this extension to the nonlinear process are 
briefly reviewed. 
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9.2 NONLINEARITY OF OCEAN WAVES 


Wave theory has developed remarkably in recent decades with the assumption of a 
wave as a stochastic process. In most cases, however, the assumption is based on infini- 
tesimal amplitudes and is valid only when the wave height is small compared with the 
wave length, and wave length is small compared with the water depth. All the quadratic 
terms of the derivatives were assumed to be small and were neglected, and the 
fundamental equations of motion were linearized and the linearized potential function 
was derived. 

In this section, the results of only a few works on the investigation of the effect of 
this neglect will be reviewed to show that the effects are indeed small. 


92.1 Second Order Spectrum of Waves by L. J. Tick and Others 
To take account of quadratic terms, L. J. Tick®™°? expanded the potential function of 
a wave (x, z, t) around the mean position of the wave z = 0 by Taylor expansion, 


g(x, z,t) = o(x,z,t) + emote) Ze ng (9.1) 
z=0 0z z=0 


and carried it to the second term. Using this expression in the fundamental equations for 
waves, through the perturbation method he expressed this velocity potential as the sum of 
the linear part ¢ ;(t) and the quadratic term @2(t) as 


P(t) = Pi(t) + G2(¢). (9.2) 


As a result, the wave height is supposed to be composed of two parts, linear and 
quadratic, corresponding to these two potentials, as 


nit) = )(t) +4 (2), (9.3) 
and expressed 
n(t) = | eTIgE (wy). (9.4) 
Here 
EfidE!?] = s%w)dw, (9.5) 
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72) = | | eho! O(@, w')dE(w)dE(w'), (9.6) 
and 
Q(w,w') = —[sgn(@), sent") (9.7) 


Then the spectrum of 7(t) was derived as 


co 


1 { [lm —Al(w —A) + AU]? 


=A) as (1g) — 2) g(1) 
sS(@)=S Orr 5 SM —A)s(A)dA 


-@ 


1 
= sV@) + sO). (9.8) 
s 


As an example, Tick took the simplified Neumann—Pierson type spectrum 
as sw) = 1.8 X 1.04w@~lw! > wo and computed this s@(w) for wo = 2,3,6 rad/sec. 
Figure 9.1 shows the results and (1/g*)s(w) is as small as several hundredths of 
s(q@) at the most. 


L. Tick® considered also a shallow water depth and derived the type of 
Q;(w,w') for this case that corresponds to Q(w,w’) in Eq. 9.6, as 


~~ LRRKE 06 w'—~=~6(w ty 
DG ON Sram ers Gin eine) aT 


(mn) 2 2 


ry2] RR 1 1, ok'+n'i¢@ _ (wr+o'y 
(w+@ ¥| BB Peano hte rena, 5) ee 
9. Sa 5 
(Ki k+1k'l k) tanh [d7(i k+1kl k')] - @Wt+o'y* 
Here d* =(h/g), k=k(w), k’ = k(w’) are the solutions of w? = /* tanh d’k’. As an 
g 


example, he showed s(w) of sw) = s'(w) + s@), for the following type spectrum, 
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Fig. 9.1. The second order spectra. 
(From Tick.®) 


sD@)=C ate, w<2 


and showed s®(w) as in Fig. 9.2. In this case, even when the water depth h = 32.2 ft, the 
effect of nonlinearity is small. 


He pointed out the necessity for computing the bispectrum to show clearly the 
quadratic effect of the waves as will be discussed in Section 9.2.2. 


On the nonlinearity of the waves M. S. Longuet-Higgins™ © and D. M. Phillips® 
discussed work along the same lines, and M. Hineno®’ referred these works and showed 
clearly the quadratic effect, as in Fig. 9.3, that was calculated for a modified Moskowitz— 
Pierson type spectrum. In Fig. 9.4, the quadratic and linear parts of the spectrum 
produced in the experimental water tank are shown. 

Hineno®’ also showed the quadratic kernel functions g(t), T2) as Fig. 9.5, that is, the 
Fourier transform of quadratic frequency responses G2(w), w2), equivalent to the 


Q(w,w’) by L. Tick, as 
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Fig. 9.2. The second order spectra s®(w) for varying depth h = 32.2, 46.4, and 72.5 ft. 
(From Tick.®) 


1 
G2(@1,@2) = ap (wt + w3) for the sum (@ + @2) frequency component 


1 
=— Oe lot = w3! for the difference frequency component. (9.10) 


Figures 9.3 and 9.4 indicate that the quadratic effect on the spectrum computations 
is very small for ordinary cases. M. Hineno® extended this result and calculated the prob- 
ability distribution of the maximum and minimum of the waves. The results are intro- 
duced in Section 12.7, and from the results we find that the effect of the nonlinearity is 
not large. 


92.2 Bispectrum of Waves 

L. Tick® suggested the need for bispectrum analysis for ocean waves. Such an anal- 
ysis was performed by K. Hasselman et al.°* as shown in Fig. 9.6. The exposition on the 
bispectrum is given in Section 11.3. Here, one bispectrum for shallow waves is given as 
an example, and it indicates that the nonlinearity is not large in this analysis either. 
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Fig. 9.3. Linear and quadratic component of wave spectrum of Pierson—Moskowitz. 
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Fig. 9.4. Linear and quadratic component of wave spectrum of 
waves produced in a towing tank. 


(From Hineno.®’) 
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Fig. 9.5. Quadratic kernel function 82(t;,7) of wave. 
(From Hineno.®”) 
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Fig. 9.6. Bispectrum of ocean wave. 


[The number —74 denotes —7 x 104 cm? sec®. Contours are drawn 
for -103, -104, -105, and —10®.] (From Hasselman, et al.®®) 
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9.3 RESPONSE OF THE BEHAVIOR OF A MARINE VEHICLE ON WAVES 


When the amplitude of oscillation in six degrees of freedom is small, the behavior 
of a marine vehicle in waves can be expressed approximately by linear equations. 


Here, for simplicity, if we assume that a floating body in a regular wave train 
f(t) = Z(w)e!' is subjected to a force bC(r) that is linear with wave height C(t), the 
equation of motion is well approximated°? by 


Q4X(t) + arx(t) + a3x(t) = E(t). (9.11) 


The response x(t) will be in the form X(w)e, 


where 


X(w)el" = X(w)lel@t#9x@)+0.0)]. (9.12) 


O¢(w), 0,(@) are the phase relations of the exciting force to the wave height, and the 
response to the exciting force, respectively. 

Generally, for a body floating on the surface of water, the coefficients a), a,b of 
the inertia term a@;x(r), the damping term, and the compulsory term, respectively, are 
generally functions of the frequency and so we find a; = Aj(@),a2 = A2(w), and 
a3 = A3(W),b = B@). 

The equation of motion, Eq. 9.11, will be 


—w7A;(w)X(w)e! + jwAx(w)X(w)e! + A3(w)X(w)e!! = Bw)Z(w)e!. (9.13) 


Therefore, the frequency response function H,¢(w) of the behavior X(w) to wave height 
Z(@) is 


nexX(@) i B@) 
ES) 7G) | a AOL) eet 


From this, expressing H,-(w) justas H(w), 
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X(w) = H(@)Z@). 
Taking the Fourier transform gives 


5a) = | h(z)6(t —t)dt. 


—-o 


Here 
l4@) = | H(w)e!*do, 


is the impulse response function. 
When the waves are expressed by a Fourier integral, 


E(t) = | Z(w)e!”'dw, 


-o 


1 
27 
the linear response x(r) will be 


x(t) = 


-o 


| X(w)e'dw. 


(Ost) 


(9.16) 


(9.17) 


(9.18) 


(9.19) 


Strictly speaking, these integrals (Eqs. 9.18 and 9.19) should be Fourier—Stieltjes 


integrals in the form of 


1 
2m 


where 
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1 
| azo», ae | e!?'dX@), 


(9.20) 


S(w)d(w +m") 


dZ(w) aZ(w') 
, (9.21) 


Sx(@)0(W +’) 


dX(w) dX(w') 


S:(@), S;(@) being the spectrum of wave height €(r) and response x(t), respectively. Here, 
however, for simplicity we use the ordinary Fourier integral form. 
From Eqs. 9.18, 9.19, and 9.13, Eq. 9.11 will be 


os | oA] X(w)e!?"'dw + | Yiodaten)xcanettdeo + | Astorx(o ela 


oO 


= | B(w)Z(w)e' dow. (9.22) 


From Eq. 9.19 


x(t—T) = | X(w)el'e@T dy. (9.23) 


1 
27 


—o 


Then setting 


| Aw )e!@"dw = k,(t) 
-% (9.24) 


| B(w)e?"*dw = I(r), 


Eq. 9.22 will be in the form of a Fourier convolution, 
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i ky(t)x(t —t)dt + | ko(t)x(t —T)dt + | k3(t)x(t —T)dt 


= ic z(t—T). (9.25) 


Actually, however, taking into account the Kramer—Kronig’s theory that connects 
the added mass and the damping for floating and oscillating bodies and also the fact that 
the restoring term a3x(r) is not a function of frequency, the equation of motion is, as 
T. F. Ogilvie” pointed out, in the form of 


{M + m( )} x(t) + | K(t—t) x(t)dt+C x(t) = f(t). (9.26) 


Here m(o) is the added mass at frequency w = ©. 


9.4 NONLINEARITY OF THE BEHAVIOR OF MARINE VEHICLES 


When the waves are moderate and the amplitudes of the motions are mild, the re- 
sponse of marine vehicles is well expressed by linear equations, as was shown in the 
preceding sections. 


The nonlinearity of ocean waves was found to be not as large as was shown in Sec- 
tion 9.2. Even when the wave itself is linear, sometimes the exciting force exerted by the 
wave can be nonlinear with respect to wave height. For example, for drifting vehicles or 
for the slowly varying behavior of moored offshore structures, the exciting force is not 
linear with the wave height because of the effects of the secondary potentials of waves. 
For rolling near the synchronous frequency, the vehicle might oscillate with such a large 
amplitude that the damping and the restoring force are not linear with the velocity or the 
displacement. For oscillatory motions, in many cases viscous resistance that is not linear 
to the velocity of motion sometimes plays a big role in addition to that of wave making 
resistance that is linear, and results as nonlinear damping. 

Usually, however, even in cases of this kind, the effect of nonlinearity is assumed to 
be small or weak in the succeeding discussions, as will be mentioned again in Chapter 11. 
Generally, under certain restrictions (as the sum of the absolute values of all kernel func- 
tions is not infinite), the nonlinear response can be expressed by 
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oo 


x(t) = ho+ | hy(t)(t—t)dt + | | h(t, T2)6(t — 71) E(t — T2)dt dt 2 


—c 


>| J J beuntte 71 )6(t —72)6(t — 73) dt dt 2dr3 


hft1,T2 . . . T%) C(t—tp)S(t—T2) . . . (t—t)dt\dt2 . . . dt; 


A 

3-8 
3 -—— 8 
12 


= | | ue | hate.te, -. « TG(E—T)SC—T2) . . . C(e—T,) 


@t\dt2 . . . Alp, (9.27) 


instead of by Eq. 9.16 for the linear case as, 


x(t) = ho+ | A(r)G(t —t)at. (9.28) 
Here in Eq. 9.27 
h,(t1,T Tn) = ! an H,(@1,@ On) 
MELE), 02 tn) TN pee te n(M1,W2 ... Wp 
X exp(i@1T; + 1W2T2 + . . . +IWzT,)dW\dW2 . . . dw, 
n=0-— oe, (9.29) 
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h,(T\,T2 . . . Tn) is called the 7» th order impulse response function and 


H,(@1,@2 . . . @,) the n th order frequency response function. The expansion of x(f) in 
Eq. 9.29 is called the functional expression or the Voltera—Wiener expression. The treat- 
ment of nonlinear processes from this stand point and the application of polyspectra in 
the analysis of the nonlinear response system is dealt with in Chapter 11. 


When the nonlinearities are expressed explicitly in the form of the equation of 
motion as 


Mx + B(x) + K(x) = F(), 


with the nonlinearity in the damping and restoring terms, there have been a few efforts to 
overcome the difficulties. In Chapter 10, these efforts will be summarized. 


Hasselman’! formulated the equations of motion for a quadratic nonlinear ship’s 
motion, expressing the wave field by surface displacement €( x,t) = ¢, and its normal 
surface velocity (df/dt) (x,t) = €2, where xX = (x;,x2) is the horizontal Cartesian coordi- 


nate vector. He derived a generalized nonlinear equation expressed in functional form 
and made it clear that, through cross—spectrum and cross—bispectrum analysis, it is possi- 
ble to get the linear and quadratic frequency responses from the data obtained from 
irregular waves on the ocean. For his formulation, the frequency components that are the 
sum and difference of two component frequencies appear to be important. His expression 
is, however, in generalized form and is not necessarily adequate for practical applications. 
Almost the same content will be explained later in scalar form in Section 11.3, for a gen- 
eral dynamic system with quadratic nonlinear characteristics. 
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CHAPTER 10 


APPROXIMATION METHODS FOR THE ANALYSIS OF NONLINEAR 
SYSTEM IN RANDOM EXCITEMENT 


10.1 INTRODUCTION 


Nonlinear vibration systems have been studied for a long time. The equivalent linea- 
rization is the most popular method for handling the weakly nonlinear system, and the 
perturbation method has been used to obtain the equivalent linear expressions. However, 
most of the studies have been concerned with systems under deterministic excitation. In 
this chapter, systems under random or stochastic excitation are treated. The equivalent 
linearization method and the perturbation method are treated independently for 
convenience of explanation. 

10.2 EQUIVALENT LINEARIZATION METHOD 


Suppose there is a weakly damped, slightly nonlinear oscillation expressed as 


X + ax + Ble + wox + koe = FO). (10.1) 


When, for example in case of rolling with moderate amplitude, the damping term 
includes the quadratic term Blxlx for viscous damping in addition to the linear damping 


ax, the restoring term includes the cubic term 4° that comes from the shape of the right- 


ing arm curve (stability curve). Generally when k20, this system is called a hard or soft 
spring oscillation system. 


In the deterministic case, or when this system is exposed to a harmonic excitation 
F(t), this nonlinear damping can be linearized with the equivalent linear damping 


8a 
Qe=at+a,=at eed) Bxo (10.2) 
LA 


using the amplitude of oscillation Xp. 


This relation was derived by Jacobson (1930),’* equating the energy dissipation by 
the damping term ax + Blilt and by @_* during one period of oscillation at amplitude Xo. 
In the same way, equating the work done by the restoring term at amplitude xo, the equiv- 


alent linear restoring coefficient we, is as shown in Fig. 10.1, 


1 
We = 0 +5 ko. (10.3) 
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Fig. 10.1. Restoring coefficient. 


However, for the stochastic case, when the exciting term is stochastic in character, 
we cannot use Egs. 10.3 and 10.2 directly. T. K. Caughey”? formulated this case as 
follows. Nonlinear oscillation can be expressed as 


X+ax+wox+y - 9(x,%,t) = fio), (10.4) 


where 7 is small, g(x,x, 7) includes all nonlinear effects, and f(r) is a random excitation. 
When the equivalent linearized damping coefficient @,, and the linearized restoring 


coefficient w2, are obtained, 


X+ ak + wegx = fi), (10.5) 


and the error e(x, x,t) is expressed by 


e(x, x,t) = (@ —Qeg)X + (Wh — Weg X +1 . B(x, 4,1). (10.6) 


Namely, eq, We, are to be set to minimize the time average 


of 

or 5 1 2) 5 

le“l= — | e(x,x, t)dt. (10.7) 
—T 
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Therefore from 


le a 
wan 0, 2[(a —Geg)X* + (Wh — W2,)xk + NxE(x,X,))° = 0 (10.8) 
OQeg 
and 
dle*| > i 
art 0, 2[(@ —Geq)x,X + (W§ — Weg)x* — NxB(x, x, = 0. (10.9) 
eq 


If this process x(r) is stationary, 


[xx] = 0. (10.10) 


Therefore from Eqs. 10.8 and 10.9 


eq =a +e, x, 1/L7I, (10.11) 
w2, = 0) +L 8G, 4, O1/L7 1. (10.12) 


If we assume ergodicity, the time average can be replaced with the ensemble aver- 
age, and therefore 


Geq= atn . Elig(x,%))/EL", (10.11°) 

Weq = H+ . Elx, g(x, x)]/Elx"]. (10.12’) 
This shows how Qeg, We, can be estimated from the original expression Eq. 10.4. When 
the form of 77 - g[x, x,t] is given, the equivalent linearized values Gg, Weg can easily be 


obtained by Eqs. 10.11’ and 10.12’. 
For example, for a Duffing type oscillation, 


Ft aitws{x+ ex = fi), (10.13) 
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g(x] =2°, 7 =€w5. When € >0, we call ita hard spring system, and when € < 0, a soft 
spring system. 
From Eq. 10.12’ 


w2g = w9|1 + €Ex*1/EL"] (10.14) 
When nonlinearity is weak, x can be approximated as Gaussian, and then 


E[x“] = 3(E Le ie (10.15) 


and thus W2q = Wl1 + 3€03]. (10.16) 


When every coefficient is linearized, the response can be obtained. If f(t) is almost 
white in the important range, E [x?] for this case can be computed as 


EL] = 02 -3e0,. (10.17) 


Here oz is the variance for linear oscillation. Eq. 10.17 shows that, whene > 0, E[x?] is 
smaller than by nearly 3€0,,. 


Applying these general results (Eqs. 10.11 and 10.12), L. A. Vassilopoulos”4 
obtained @-, and We, for ship’s rolling, expressed as 


X+ax+Pldixrt+ op xt+ek-r =fir), (10.18) 

as 
eq = a+, = a+ /8/n -B Ox (10.19) 
Wg = w+ [302] -k. (10.20) 


Further, assuming that the damping is small and the spectrum of the wave slopes is 


flat in the important range for rolling, i.e., 0; = @o0; , he derived 
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a+a-=at+/8/x-B -wodx. (10.21) 
Comparing this expression with Eq. 10.2 for the deterministic case gives 


£ 1.6 


x0 = (SJ 085 


Ga— 20%. (10.22) 


namely for our interest, xo appeared almost equal to the significant amplitude for the 
stochastic case. 


P. Kaplan” used this general result and calculated the ship’s rolling for the same 
example that this author solved by the perturbation method as shown in the next section. 
Kaplan found that the result checked very well with this author’s result. 


10.3 PERTURBATION METHOD 


10.3.1 Trial for Ship’s Rolling 


This author showed”? the results of the computation of nonlinear rolling using the 
perturbation method. The effect of nonlinear damping was investigated first. One degree 
of freedom rolling is expressed as 


Ip +N\d + No)? +Kig = M, (10.23) 


d + 2ad + Bidld + wih = mit). (10.24) 


Starting with a 0 order approximation ¢o (when f = 0), that is linear, 


do + 2abo + WewWo = mit), (10.25) 


go= | hg ym(t —T) m(t)at = | hggm(T) m(t—T)at. (10.26) 


Here hg,m(T) is the impulse response of the 0 order, namely linear approximation @o to the 
exciting moment m(t), 
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1 , 
hg mT) = aS | Hg yn) Cate (10.27) 


Frequency response Hg ,(@) is easily obtained by Eq. 10.25. 


Then, when the nonlinear damping term is shifted to the right-hand side of Eq. 
10.24 and ¢o is used in the nonlinear term, the first approximation $, is expressed by 


$1 + 2ab) + wi. = mt) —Bipoldo. (10.28) 


As the left-hand side of Eq. 10.28 is in the same form with that of Eq. 10.25, using 
hgom(T) as the impulse response function gives 


gi= | hgom(t—K)|m(L) —Blbow)ipow)} du, (10.29) 
= go(t)-¢1 (0). (10.29”) 


Since a and are small, the convergence of this approximation is assumed. Then, the 
correlation function is 


Ro.9,0) = El{Pole + 1) - 91 (t+ D{Golt) - $1 @} *] 
(* Shows the conjugate) 
= Elpolt + tpd'(t)] — 2 Re {Elpile + 1)1(0)) } + ElGie + HG 1* 1, 
(10.30) 


here Re { - } indicates to take the real part of a function {-}. 


The spectrum is manipulated by taking the Fourier transformation of Rg,g,(T), 
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2 
56:9) = Soop) — 2 ‘| 7 2boP Soop) @- Im|Hon(o)| 


2 2 2 4 
+ BlHgm(@)! =o} OSoop®) + == Sapo) | .(10.31) 
Go 


where Jm {- } indicates the imaginary part of a function {-}. 


Here 


S¢ob\M1) S$opo(M2) SGop.(@ —@1- @2)dw\dw2. (10.32) 


ae 


S obo (@) = i 


In the computation of these values, the expected values E[&- 7 - byl] and E[&,1E|-7 - ll, 
that include the absolute values [&|, 7] must be calculated. Here € and 7 follow the two 
variables’ Gaussian distribution, 


o2E* — 200 +02n° 


1 
IED SS 
271020 ¥ 1-07 20707(1 -@7) 


(ce is the correlation). 


The expected values of the two variables such as E[& n°] E[é ZI n°] have been computed 
by Isserlis,’° a long time ago, but no reference was found on the expected values of the 
products of variance that include the absolute values as E[&,7, byl], E[E - El-7 - byl). 
These were calculated by this author and were found as 


BIE-n- in) =,/2 Q 0407, (10.34) 
107 || = EG 
E[E - \Eln - inl] = i oeso4d x SVS IVE Mae 
[5 - Sl - il] = Vl1-@-3@+2(1+ 20°) X tan eos o 
(10.35) 
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A numerical example for a ship model, with wo = 3.85, (@/wo) = 0.06705, 


B = 0.08, and waves with a Neumann type spectrum and a peak at wo, is shown in Figs. 


10.2 — 10.5. Figure 10.5 shows the overall effect of nonlinear damping, and Fig. 10.4 
shows the effects of nonlinearity on the term of Eq. 10.31, especially the double convolu- 


tion Sg¢g(w) of the spectrum Sgg(w), shown by Eq. 10.32. 


In the same way, the author also calculated the effect of a nonlinear restoring force 
expressed as 


Ip + NO + (Kip + K3*) = M, (10.36) 


d + 2ad + wip + k3h? = mit). (10.37) 


Starting with a 0 approximation go, for k3 = 0, the spectrum of the first order 


approximation @; when k3+ 0, is 


Sob) = Soop (@) — kz + 2RelHggm(@) - 305, 59.9()] 


+ UH gn(00)?|904 Spp() + 60 S51, (@)}. (10.38) 


Here 


oo 


Sy Oe | | Shap Sbop(O2)Sbop(@ —W1-—@2)dw dw». (10.39) 


—co —0O 


10.3.2 General Formulation of the Perturbation Method 


S. H. Crandall’’ formulated a general approach to the case with a nonlinear 
restoring term by the perturbation method, as follows. 


The equation of motion is expressed as 


X + 2aX + wp [X + €9(X)] =f) (10.40) 


where € has a small value. Then X can be expanded in terms of powers of € 
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Fig. 10.4. Convolution of spectrum S,;. 
(From Yamanouchi???) 
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Fig. 10.5. Computed nonlinear spectrum of rolling. 
(From Yamanouchi?’) 
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MSD OUM SEDO s50 2 (10.41) 


Substituting Eq. 10.41 in Eq. 10.40, gives 


[Xo + 2aXo + wiXo —fit)] + €[X1 + 2aX) + WX) + WHe(Xo)] 


+ €7[X> + 2aX> + wiX2+ wpe(X)] +... 


0. (10.42) 


This equation is independent of €, so each term with powers of € should independently 
be equal to zero, 


Xo + 2aXo + weXo = fit) 
X, + 2aX, + woX, = —we(Xo) (10.43) 


Xp + 2aX2 + wEX2 = — wog(K1) = — w9X 1g" (Xo). 


Usually the first approximation X = Xp + €X, is used. From Eq. 9.16 


oo 


Xo = | h(t) fit —t)dt. Here Xp is the linear response. Accordingly, from the first equation 


—-o 


of Eq. 10.43 


h(t) = —== sin («3 = a?)?r (10.44) 

{@3-2) 

Then from the second equation of Eq. 10.43 
X(t) = (1% | h(t)gXo(t—T)dt , (10.45) 


which is the expression for the first order approximation. 
Here some expectations are 
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Xj = Xo +€X, 


E[X7] = E[X6] + 2€ E[XoX1] 


A(ty) A(t) Efft —t)flt—1)}dt, dr. (10.46) 


_— 8 


and E[X3] = | 


8 


When f(r) is stationary, 
RAt) = Efft) fit +7). 


For example, for Duffing type oscillations as 


X + 20X + wa[X + €X?] = fir), (10.47) 
in Eq. 10.40, 
g(X) = X?. (10.48) 


Therefore from Eq. 10.45, 


E[XoX1] = - 03 | h(t)E[Xo(0)X9(t —7) dt 


=- 0 | h(c)dt | h(ty)aty | h(t2)at | h(t3)at3 | h(t4)dta 


—c —o 


X Efft-—tp)ftt -—T-To)flt -T-T3) ft -—T-T,)]. 


When Z(r) is a zero mean stationary random process, the following relations apply: 
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E[Z(t1)] =0, ElZ(t1)Z(t2)] = R2t2-t1), 
E[Z(t1)Z(t2)Z(t3)] = 0, 
E[Z(t1)Z(t2)Z(t3)Z(t4)] = R2(t1 — t2)R2(t3 — t4) 


+ R(t; — t3)R2(t2 — t4) + Ro(t) — t4)R2(t2 — 13). 


(10.49) 


From these relations and the assumption that the external force is white in the important 
range of frequency in which the response shows a peak, 


1 
Rt) = 5 Wod() 
(0(t) is Dirac’s function.) Using these relations in the second equation of Eq. 10.46 gives 


E[X?] = 0% -—3e04. (10.50) 


It is interesting that the same result was derived by the equivalent linearization method, 


Eq. 10.17. 
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CHAPTER 11 
VOLTERA EXPANSION AND APPLICATION OF POLYSPECTRA 


11.1 VOLTERA—WIENER EXPRESSION 


Suppose there is a weakly nonlinear response process Y(t) which is an output of a 
linear input process X(t). Both Y(t) and X(r) are stationary up to an appropriate order. 
Then when the sum of the absolute values of all kernel functions is not infinite, Y(t) can 
be expressed as 


foe) 


Y(t) = | 1 X(t—T) adt+ | 


—o 


h2(t1,T2) X(t—T) X(t—T) atyat2 


SS 3 


8 


loo} 


-| | | hace, 12.23) X(t—T,) X(t—T2) X(t—73) dt\dt2 dt3- - - (11.1) 


-o 


n=0 


Dy | |. ae | ttes.t- > + Ty) X(f—-T1) X(t—T2) > - -X(C—-T) 


dT\dt2- - - dtp. (11.1’) 


Here h,(tT2- - -T,) is areal function of n variables —~© <T;< +0, j =1,2,---n 
and is called nth kernel function. All kernels are assumed to be smooth, absolutely inte- 
gratable, and to possess Fourier transforms. Besides, all these are supposed to be time 
invariant. When, for any 7; < 0,/,(T),T2,: - -T,) = 0, then the lower limit of these inte- 
grals can be 0. For n = 0, hy is the value of Y(t) when X(t) = 0. So we can include n = 0, 
generally; otherwise we assume h, = 0. Whenn = 1, Y(t) will be the linear system that 
has been treated in Part I and I and as Eq. 9.16 


Y(t) = | h(t) X(t—1) at. (11.2) 


Equation 11.1 is assumed to be a kind of Taylor expansion of Y(t) around the linear 
process expressed by Eq. 11.2. Accordingly, the terms form = 2 are regarded as modify- 
ing small terms. 


If we change the variables, 
Os yY | | Oi | tate—nit—ta, » + £-T,) X(T) X(T2)- > -XCn) 


at\dt.-- - aT, . (11.3) 


Equation 11.1 or 11.3 is called the Voltera—Wiener or functional expression. 
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11.2 HIGHER ORDER RESPONSE FUNCTION, 
hy(%j,T2,° * - Tn); An(W1,@2,- > -Wn) 


When h,(T;,72,° * :T,) are smooth and absolutely integratable, the nth order Fourier 
pair exists as was stated at the beginning, 


1 
hy(T1,T2,° ° Tr) = ope |. ao | expifon+0: T2 +++ ++Op Tn| H,(@1,@2,° - Mn) 
—-@ —_o 


dw, aW2---dW,, (11.4) 


H,(@1,@2,° + -@n) = | 26 | expi-ivo, TT +2 T+: - -+@r,Tn)] 
lO ACs, ? © Sth) Gim@rb)e © orm (US) 


h,(T1,T2,° - -T,) are real functions and are symmetrical fort,,T2-- -T,, 


thus 


h,(T1,T2,° > - Tr) = hn(02,T1,° - Tr) =- - + =Mn(t2,73,- > - t,T1) (11.6) 
H,(@1,@2,° > - Wn) = H,(@2,@1,° > -@p)=- > + = Ay(W2,03,- - - Wp, @}). (11.7) 
If we regard Y(t) as the output and X(t) as the input process, h,(T),T2,- - - Tn) 18 


called the nth order impulse response function and H,(@,@2,- - :@,) is the nth order 
frequency response function. 

When the processes are not continuous but discrete, then the Voltera—Wiener 
expression for the nonlinear response Y, and linear input X, is 


Y, = yy 8u Xpu + y y 8nv KGa, Xtv 
uUu=o u=ov=o0 
Hoi Mita Xen, Nebr Xa ce in (11.8) 


u=Ov=0W=0 


= yi >: ; use tu, Xt Xe? * + Xtuys (11.8’) 
od tl 


n 


and the following discrete Fourier pair is assumed to exist: 
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co eo io) 
G(@,, 2, = Ure On) = yy y Pate a » Su, Uy: ++ Uy EqOreir@ au aa +0 Hn) 
= =o u,=0 


pee (11.9) 
cL It 
1 . 
Sujuy O97 = (27r)” | oa | G(@, OD a Wn) ellOilrt@2ot- + +O Un) 
dw\dw.----- dw, (11.10) 


For convenience the expressions for a continuous process will be used in this 
chapter unless otherwise stated. 


11.3 SECOND ORDER NONLINEAR PROCESS 

BISPECTRUM, CROSS BISPECTRUM 

For example in Eq. 11.1, the quadratic nonlinear characteristics of Y(t) is expressed 
by n = 2, and the terms n > 3 can be neglected. Then Eq. 11.1 becomes 


Y(t) = | hy(t)X(t—T)dt + | | h2(t1,T2) X(t—7T1) X(t—T2) dtydtz, (11.11) 


which corresponds to Eq. 11.2 for a linear process. As the expected values of the second 
moment, the auto or cross covariance functions Rxx(T), Ryy(t), and Ryy(t), and their 
Fourier transforms, the auto or cross spectra syx(), syy(w), and syx(w) played big roles 
in the analysis of the responses of linear systems, the expected values of the third 
moments Ryxxx(T}1,7T2), Ryyy(t1,T2), Ryxy(t},T2), and their double Fourier transforms 
Sxxx(@1,@2), Syyy(@1,@2), Syxx(@),@2), are important in the analysis of second order 
nonlinear processes. The third moment correlation functions are, for example, 


Ryyy(1, T2) = E[{¥(t + 71) — my|[¥(¢ + t2) — my|[¥(¢) — my} (11.12) 
Ryxy(t,T2) = E[X(t +1) X(t + t2)[Y(0) — my]]. (11.13) 


Here E[X(t)] = 0 is assumed. Their double Fourier transforms are called the bispectrum 
Syyy(@ 1, @2) or the cross bispectrum syxy(@ 1, @2), 


Syyy(@ 1,2) = Ryyy( 1, T2) exp[— i(@ jT1 + W2T2)|dt,dt2 ~— (11.14) 


a 


SXx¥(W1,@2) = Ryxy(T1,T2) exp[- i(@ 171 + W2T2)) dtydt2. (11.15) 


Bat 


fl 
fi 


The inverse transforms are 
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Syyy(@ 1,@2) exp[i(@1T, + W2T2)] dw dw (11.16) 


Ryyy(1, 72) = SXXY(W 1,2) exp[i(W 1T1 + W2T2)] dw\dw2. (11.17) 
When the input process X(r) is a Gaussian process, then 

Ryyx(T1,T2) = E[X(t+7,) X(t+T2) X(t] = 0. (11.18) 
Therefore 

Sxxx(@1,@2) = 0, (11.19) 
The bispectrum does not exist for a linear Gaussian process. 
For a quadratic (second order nonlinear) process Y(t), Ryyy(t1,T2) = 0, 

Rxxy(t1,T2) # O and the bispectrum exists as shown by Eq. 11.14 and Eq. 11.15. 


Now, assuming Y(f) is a stationary process and its Fourier—Stieltjes integral is 
foe) 


{Y(t) — my} = | dZ(w) exp(iw ft), (11.20) 
then {Y(t +1) — my} = | dZ(w 1) exp(iw jt + iw 4,7), 


{Y(t +72) — my} = | daZ(w2) exp(iw rt + iw2T2). 


Therefore 


Ryyy(t1, 72) = E[{(Y(t + 71) — my]{¥(¢ + t2) — my}{¥@) — my}] 


fo) 


= | | | expt, +W2+3)t] - exp[i(@1T1 + @2T2) 
E[dZ(w1)dZ(w2)dz(w3)] (11.21) 


[w is expressed as w3]. 


Because of its stationarity, Ryyy(T),T2) is a function only of T;,72 ; it is not a func- 
tion of ¢ but is independent of r. Accordingly, the integral has a value only when 


@,+@2+@3 =0 or 03 =-@)-@2 (11.22) 
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comparing Eq. 11.21 with Eq. 11.16, and taking into account the relation 


oO 


1 
= | exp[i(@1 + @2 + w3)tdt = 0(@; + w2 + @3)], 


—-o 


| E[dZ(w1) dZ(w2) dZ(w3)] 6(@1 + @2 + W3)dw3 = Syyy(W1, 2) 
0 dw \dw2. (11.23) 
This shows that E[dZ(@,) dZ(w2) dZ(w3)] must be 0 except along the line 


@,1+W 2 +W3=0. This corresponds to the fact that, for the linear case, 


Rxx(t) = | | expo +0"rexpior - E[dZ(w) - dZw"| 


—-@® —CO 
ao 


= | expio T - Sxx(w) dw, (11.24) 


and | Fraziw 220" y0w +w')dw' = sxx(w)d(w). 


By the same logic that because of stationarity, Ryx(t) should be independent of r, there- 
fore E[dZ(w) dZ(w’)] must be zero except when w +’ = 0, i.e., only along the line 
@+o'=0 andw’=-wo. 
Because of the symmetry of Ryyx(T1,T2), 
Rxxx(T1,T2) = Rxxx(T2, 71) = Rxxx- 11, T2 — 71) = Rxxx(t2 -71,—-T1) 


= Ryxx(—T2, 7] —T2) = Ryxx(T1 — T2,-T2) (11.25) 
the spectra also have symmetry, 
SXXX(W 1, 02) = Sxxx(W2, 01) = Sxxx(W1,— W1 — W2) = Sxxx(-@1 — 2,01) 
= Sxxx(W2,-—@1 —W2) = Sxxx(—@1 —@2, 2) (11.26) 
and SXxx(@1,@2) = S * (—@j,-—@2). (11.27) 


Accordingly, when we think of the bispectrum sxxx(@1, #2) we must consider the 
third frequency w3, besides w, and w2, that makes 


@1+@2+03=0, W3=—-@|-@2. 


From Eq. 11.26, the bispectrum is found”® to be the same for six permutations for two of 
the three frequencies w 1,2, and w3. Twelve bispectra, including the conjugate bispectra 
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show the same modulus as is shown in Fig. 11.1 for G) to @). These bispectra are repre- 


sented by a bispectrum in the first octant G) where |w3| = @ = w2. The bispectrum is 
shown by the modulus and the phase angle. 


Ea @, + @2+ @; =0 


Fig. 11.1. Symmetric character of bispectrum. 
(From Yamanouchi and Ohtsu.78) 


The points marked O in sub—octants Q), G): ©) (9). and (10) show the same 
bispectra, equal to Syxx(@1, @2) at mark O in octant Q), and the bispectra at points Xx in 


the six sub—octants @). ), @; (8), @), and @2) show its conjugate. Also in Fig. 
11.1, the segments of line in each sub—octant are shown marked by short lines +, +f, +t 
that correspond to the segmented line AB +, BC+, CD + in octant @) that shows the 
same bispectrum. At a fixed frequency wz, in Fig. 11.1, the bispectrum along the segment 
BA (wz = 3) shows interference with the two components at the smaller frequencies 

@ and @, that makes w; + w2 = Wg = @3; the spectrum along the segment 

BC (wz = @) shows interference with one smaller frequency component w and one 
higher frequency component w3(w3 = w2 + w)); and the bispectrum along the segment 
CD (wg = @2) shows interference with the components at two higher frequencies 

W3(W3 = W, + W2) and @;. The hexagonal boundary shape in Fig. 11.1 shows the 
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boundary of the frequency region in which the bispectrum can be calculated for the data 
sampled by interval At . 


Figure 11.2 is an example of the bispectrum of the rolling of an actual ship on the 
sea, reported by Y. Yamanouchi and K. Ohtsu.’*:’° The bispectrum in Fig. 11.2 shows 
higher values along a similarly shaped segment ABCD in Fig. 11.1 with w, at the fre- 
quency at which the linear spectrum, drawn down the side of the w, axis, shows the peak 
value, that implies the nonlinear interference is higher along these shaped segment lines, 
as ABCD. 


ROLLING BISPECTRUM 
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Fig. 11.2. An example of bispectrum of rolling of a ship on the sea. 
(From Yamanouchi and Ohtsu, 78:79 (1972).) 
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As has been formulated in Eqs. 11.12 and 11.6 
E[¥(t)— my|[¥(¢ + 11) — my][¥ (ri + 72) — my] = Ryyy(t1, 72) 


S(@1, @2) exp[+ 1(@1T1 + W2T2)] dw dw 2 


8 


8 


thus 
oc co 


E|[¥@)—my}*| = Ryvv(0, 0) = | | stco:,2ddo da (11.28) 


—-O —-O 


The bispectrum shows the contribution of the third moment of frequency components at 
three frequencies @),@2, and 3, where w; + W2 + W3 = O are satisfied. When square 
mean, cubic mean, and fourth power mean are expressed as 


El [ys - my} | Oe El{y(e) = my}*| =, El{y(e) - my|*| =u, 


the statistical values called skewness s, and peakedness p are defined as 


3 
s=—_, A (11.29) 
(o?) 
4 
u 
=—., (11.30 
PU G2) 


When Y(t) is a Gaussian (not nonlinear) process, “> =0, «4 = 30%. Therefore, 
s=0, p=3. By the values of s and p, we can show the extent of nonlinearity of the 
process. ”? 


11.4 CHARACTERS OF QUADRATIC RESPONSE TO 
GAUSSIAN INPUT PROCESS 


As already was shown in Eq. 11.2, when X(t), the input process is a Gaussian linear 
process with E[X(rt)] = 0, 


Ree Exe +1) X(t+™) x(o] = (11.31) 


SXxx(@1,@2) = 0. (11.32) 


Several statistical relations of the quadratic nonlinear output Y(t) to input X(r) and their 
derivations will be summarized as follows: 


1. Mean of Y(t), E[Y(t)] = my. 
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From Eq. 11.11, and as E[X(t-7)] = 0, 


8 


my = E[Y(t)] = | ho(t1,T2) E[X(t—11) -X(t—T2)] at - dt2 


= | | h2(t1,T2) Rxx(t2—71) at\dt2 (11.33) 


Sxx(@) exp iw(T2 —T) at at at 


fh) 


= | frcae} 


= | | tote, m)exp lor -wr) dt \at | Sxx(@) dw 


—-o foo} 


= | Hx(o,-w)ox (w) dw (LE3 35) 


because | | ho(t1, T2) exp i(Wt2 — WT1)dt, dt2 = H2(w,-w) 


—-O —-O 


2. Cross correlation Ryx(tT); cross spectrum syy(@). 
For Ryx(t) = E[{¥(¢) — my} - X(t—1)] = E[Y(t) -X(¢—1)|—my E[X(t-7)], 
inserting Eq. 11.11 and E[X(t—t)] =0, yields 


= | Ay(t1)E|X(t —11) X(t -1) dr = | mucenrte—ner (11.34) 


—-o 


Therefore 
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1 1 
SAMO) = | Reade) ac ==— | | rucenRe—zp edt at 


1 ? t 
oe | hy(t)-e "dt ,- | Re-r) e OT —T1) gy 
= H;(@)sxx(w) (11.35) 
and = Hy) = syx(@)/sxx(w). (11.36) 


Eq. 11.36 shows even when Y(r) includes the nonlinear component, if the input X(¢) 
was a Gaussian process, the linear response frequency function H (w) is expressed by the 
ratio of syy (w) and sxy (w) as in the purely linear case. 

3. Autocovariance Ryy (7); linear spectrum syy (w). 

As in the linear process, the Fourier transform of second order moment or the auto- 
covariance function is called the linear spectrum, 

Ryy(t) = E[Y(t) — my] [Y(t + 7) — my] 
= E[Y(t)- Y(t+1)] — ms. (11.37) 
Here by the use of the functional expression of Y(t) and Y(t +t) as Eq. 11.11 and appro- 
priate variables, 


Ryy(t) = | hy(s1) hy(r1) E[X(¢—5s)) -X(t+t—-11)] dsydry 


8-—, 8 


ioe] 

: | 
—-o 
Generally, when X,, X2, X3, X4 are the probability variables that are from a joint Gaussian 


distribution, then 


E[X, X2 X3 X4] = Miz Maa4+M 13 Mo4+ My4 M23. (11.39) 


| | ho(uy, u2)h2(v1, v2) E[X(t—uy) X(t—uz) X(t+ T-Vv1) X(¢+T—v2)] 


_-@® —-o 


dujduzdv,dv7 — mé. (11.38) 


Here 
Miz = E[X; - Xx). 


From this relation, 
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E[X(t—uy,) X(t—uz) X(t+T-¥) X(t+T—-V2)] 
= Ryx(u, — U2) Ryex(v1) —V¥2) + Ryx(t + uy —V1) Ryx(t + U2 -— V2) 


+ Ryx(tT + Uy —V2) Ryx(T + U2 -¥}). (11.39°) 


When this expression is inserted in Eq. 11.38, the term that comes from the first 
term of Eq. 11.39 is the same as m? in Eq. 11.33 and it cancels with the last term of Eq. 


11.38, — my. 


Then from the remainder 


Ryy({t) = J J h(s}) h(r1) Rxx(t + 81-11) 
+ ! J J J h2(uy, U2) h2(v1,¥2) Ryxx(t + uy—V1) Rxx(t + uz — v2) 


du,duzdv\dv2z 


h2(uy, u2)h2(V1, V2)Rxxx(T + Uy — V2)Rxx(T + U2 — V1) 


+ 
=< 
ie 
8 
se 


du,duzdv,dvz. (11.40) 


The terms h2(u;, U2), h2(vj, V2) are symmetrical for u;, v2 and v;, v2, and the scope of the 
integral is the same as for — © — + © for both the last two terms. Accordingly, the sec- 
ond and third terms are equal. Therefore, 


foo) 


Syy(@) == | Ryy(t) ec dt 


—-o 


1 
= Hy(w)| ? syxx(w) + | [x | expt-ilo-(o: +02) at 


—o 


X 2H2 * (@1,@2) H2(W1,02) SxA1)5xx (W2) dwidwp. (11.41) 


Here, using the relation 


x | ox [i{(@1+@2)-a@}t] dr =6{((@;+@2)-o} (11.42) 


6: Dirac’s delta function. 
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in Eq. 11.41 gives 


Syy(W) = lH\(w)I*sxx(w) 


oOo w 


+2 | | H2(@1, 02) sxx(@1) Sxx(@2)6[@1 — @ — @2)}dw dw? 


—-O -—-O 


= H,(w)?sxx(w) + 2 | \H>(@ — ©, @2)°Sxx(@ — W2)sxx(@2)dw2. (11.43) 


This shows that the linear spectrum syy(w) is the sum of the spectrum of the linear 
part IH, (w)P?sxx(@) and the modifying term in Eq. 11.43 defined by the second nonlinear 
response function H2(w —w2, 2) and the product of two linear spectra syx(w —@2) and 
Sxx(@2). 


4. Cross correlation of third moment Ryxx; cross bispectrum, syxx (w), and the 
second order frequency response function Hz (w),@2) . 


The cross correlation of the third moment, for example Ryxx (T),T2), 1S 


Ryxx(T1, T2) 


E{{Y(t)— my} X(t—1) X(t-12)] 
E[Y(t)X(t—1) X(t—12)] — myE[X(t—11) X(t—1)]. (11.44) 


Here expressing Y(t) by its functional expression Eq. 11.11 and using the same relation as 
Eg. 11.39’ and Eq. 11.33 for my gives 


Ryxx(t1,T2) = | | ho(uy, U2) E[X(t—uy) X(t—u2) X(t—T1) X(t—T2)] dujduz 


—my Rxx(T —7T2) 


= | | ho(uy, u2)[Rxx(u1 — U2) - Rxx(t1 —T2) 


+ Ryx(t;-—u 1) Ryx(t2 — U2) + Rxx(t2— uy) Rxx(t1 —uz)] dujduz 


co oo 


= | | h2(v1, v2) Rxx(V1, v2) dvidv2 Rxx(T1 —T2). (11.45) 


In this equation, the first term and the last term are the same and cancel each other. The 
second and the third terms are the same, as h(u, uz) is symmetrical with u, and v2, and 
the range of integrals is from — © — + © for both. 
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Therefore 
Ryxx(T1, T2) = 2 | | hp(uj, U2) Rxx(t1 — uy )Rxx(T2 — u2) du;duz (11.46) 


and oo 
2 
1 
Syxx(@1,@2) = | — | | Rexx. t2) exp\—ilo n+ w2t9) dt\dt2 


27 
2 E 
= | | h2(uy, uz) Rxx(t1 — uy) Rxx(t2 — u2) exp[— 1(@1T + W2T2)] 
—co —o~ at dt (11.47) 


SYXx(@1,@2) = 2 H2(@ 1,2) Sxx(@1) - Sxx(@2) (11.48) 
SYXx(@1, @2) 
2sxx(@1) sxx(@2) 


Here with the variables w1,@ 2 into @;’,w2' inverted by 


H>(@ 1, @2) = (11.49) 


1 
@,= 3 1 + @2) 
@1-@2 =] 


Dans aoe (11.50) 


1 , U 


then 


SYXX(@1, @2) = Syxx(@1—@2,W1 + W2) = Syxx(@1,02) J me ay . (11.51) 
2. 


’ 


Since Jacobian 


Le tgaek 
Aste me = = s 5 = > (11.52) 
Py) 
therefore 
Syxx(@1, @2) = Syxx(@1,@2) X : (11.53) 
and 2 syxx(@1—@2,@1 + @2) = Syxx(@1, @2). (11.54) 


Substituting into Eq. 11.49 gives 
Ss @1—@2,0,+@2) 
A2(@1, 2) = Syxx(@1—@2,01+@2) (11.55) 
SxXx(@1) Sxx(2) 
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In the same way, if we use X?(t) instead of the output process Y(t) and calculate the cross 
bispectrum Sy2xy(W1,@2) , 


Ryoygdt1. 2) = El[X%0) — my 2} -X(e—1y)(¢-72)] 
= E[X()X(t) X@—1) X¢—1)] — my EXC) XC-79)] 
= Ryx(0) Rxx(t2—71) + 2Rxx(T1) Ryx(t2) — my 2Ryx(T2 — 71) 
= 2 Rxx(t1) Rxx(t2), (11.56) 


2, 
(27)? 


thus Sx 2xx (M1, @2) = | | Ryxx(71)Rxx(T2) exp[- i{w yt, + @2T>}] dt \dt7 


= 2 sxx(@1)Sxx(w2). (11.57) 
Inserting into Eq. 11.49 gives 
SYXX(W 1, @ 
H2(@1,@2) = ue) (11.58) 
Sx2xx(M 1,02) 
and inverting the variables as in Eq. 11.55 gives 
SYXX(W1 — @2,@1 + 2) 
le KONO) Fre (11.59) 


Sy2xx(W1—W2,W1 + W2) 


Both Eqs. 11.49 or 11.55 and Eq. 11.58 or 11.59 can be used for the computation of 
H>(@,@ 2). However in actual computation, due to the consideration of the window, 
Eq. 11.58 or 11.59 is much better than Eq. 11.49 or Eq. 11.55. 

5. Application of bispectrum analysis. 

J. F. Dalzell9> 56 57, n4 58 applied these theoretical relations (as compiled above in 


items 1 through 4) to the problem of added resistance of ships in waves. The quadratic 
process Y(r) is the resistance D(t) of ships in waves and the input process x(t) is the wave 


height 7(t) in Eq. 11.1’ (when n =0 — 2) and ho = Do, as 


D(t) = Do+ | rue W(t —T)at + | | totes.22) n(t—T1) H(t—T2) at\dt2. 


Then Egs. 11.33 and 11.33’ are the expression of mean added resistance in waves 


foe) 


E(D()] = | Hx@,-)s, ,(@) do, (11.60) 


-o 


and the linear and nonlinear response functions of resistance to waves are Eq. 11.36, 
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Eq. 11.55, and Eg. 11.59 as 
H\(@) = spy(@)/S, ,(@) 


H(@1,@2) = = 
Sy n(® Sqn(@2) Sy? mm (@1—@2,@1 + @2) 


as a special case of Eq. 11.60. Therefore, 
Sdy (20, 0) if SDyy(2w, 0) 


Hx(@,-@) = (Qw.0) 


syn(w)) ‘ Sean 


SD yn (@1—@2,01 +2) Spy (W1—-2,01 + @2) 


(11.61) 


(11.62) 


(11.63) 


Dalzell performed a series of tests in a towing tank on resistance increase on en- 
countering head waves, with sea states A, B, and C, where the significant wave heights 
are in the ratio of 1:2:4. The results were analyzed and the applicability of these theories 


was made clear. 


Figure 11.3 shows the linear frequency response function [H,(w) = G(o) in 


Dalzell’s expression] obtained by Eq. 11.61 in real and imaginary parts, and Fig. 11.4 
shows the impulse response function A(T) that is the Fourier transform of H(@) in the 


discrete form L,, 
iy{( = Le d(- KAD}, 
At being the sampling time interval. 
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Fig. 11.3. Linear resistance frequency response. (From Dalzell.°°) 
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(11.64) 


0.06 


Ly 


0.02 


illu. 


“il 
0 | ae 


! 


Fig. 11.4. Coefficients L, of linear resistance impulse response function. 
(From Dalzell.55) 

Here ag, is the nondimensional encounter frequency o, = w,./w,, where w, is the 
frequency of the wave with length equal to the model ship length L, i.e., where 
@, = | 27g/L. The added resistance and wave height are also nondimensionalized by 
the displacement and the length of the model. Figure 11.5 shows the special case of 
H7(@,@2) whenw; = w,@2 =—w. The term H>(w,—w) is expressed by Eq. 11.63, and 
the results obtained in regular wave tests are also shown. The agreement is excellent. 
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© 40 
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0 
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Fig. 11.5. Estimate of G2(o.,-0.) from cross—bispectral analysis, Fn = 0.15. 
(From Daizell.5) 
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Figure 11.6 shows an example of the three—dimensional plot of the modulus of a 
cross bispectrum of wave—wave resistance Spyy(W -@2). Fig. 11.7 shows the plot of the 


real part of the same cross bispectrum. 


|Cp(Q4, 22)I 


Fig. 11.6. lsometric plot of modulus of wave—wave resistance cross bispectrum, 
sea state B, Fn = 0.15. 


(From Dalzell.55) 
The axis (2, 025 used here is Q) = w;—W 2, Q2 =w, +>. Figure 11.8 shows an 
example of the real part of H2(w , 2) obtained by Eq. 11.62, and Fig. 11.9 shows the 
second order impulse response function h2(T1,T2) in discrete form Q jk as 


h2(T1,T2) > Oj 6, —jAt) 6 2 — kAt) (11.65) 
in the form of weighting functions. These values were obtained from the data in sea state 


A. Using these weighting functions L; (Fig. 11.4) and Q;, (Fig. 11.9), obtained from the 


test data for sea state A, the added resistance was calculated by 
P 


Ue P 
Day= Y Lin-j+ Y YL O«gen-jn-wH (11.66) 


j=-m Ui=—P = =P 


for sea states B and C, where the discrete readings 7(n —j),7(n — k) of the time history of 
wave height 7(t) for sea states B and C were used. The added resistance thus calculated 
was compared with the real time history of added resistance in sea states B and C. The 
results for sea state C are shown in Fig. 11.10 as an example. The agreement of this com- 
puted time history with the actually observed time history is surprisingly good. Here, two 
digital filters, as shown in Fig. 11.11, were used to get rid of the effect of the finite length 
of the digitized impulse response functions. 
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Fig. 11.7. Real part of wave—wave resistance cross spectrum, sea state B, 
Fn = 0.15 (normalization const. = 3.76 x10-). 


(From Dalzell.*5) 
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Fig. 11.10. Time history of waves and resistance observed and computed. 
(From Dalzell.*5) 
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Fig. 11.11. Amplitude response of two digital filters. 
(From Dailzell.55) 


11.5 APPLICATION OF THE HIGHER ORDER POLYSPECTRA 


When we have to take into account nonlinearities of higher order than quadratic, we 
need to consider the polyspectra of higher order than the bispectrum. For example, when 
Y(t) is expressed by 


oO 


Y(t) = | hy(r) X(t—r) drt | | hp(uy, U2) X(t—uy) X(t—Uuz)du,duz 


—co —_ 


oP | | | h3(q1, 92,93) X(t— qi) X(t— 92) X(t—9q3)dqidq2rdq3, (11.67) 


we have to consider the fourth moment correlation functions and their three—dimensional 
Fourier transforms, which are called trispectra. In this case, as we did for the quadratic 
case, we can calculate the statistical values as summarized in the following paragraph. 


There, the variables X;, X2 ,- - - X, are supposed to follow the joint Gaussian distri- 
bution, with £{X,} = 0, then as 
E[X] = 0, E[X,X2X3] = 0, E[XiX2-- -X5]=0, ---- (11.68) 


all of the odd moments are zero. Further, as shown in Eq. 11.39 
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E[X 1 X2X3X4] = My2M3q + M13M24 + M14M23, (11.69) 


E[XX2X3X4X5X6] = M12M34M56 + My2M35Ma6 + M1 2M 36M, 
+ M13M24Ms5¢6 + M13M25M36 + M13M26Mas 
+ M14M73M56 + M14M25M36 + M14M26M35 


+ M15M34M6 + M15M32Mae + M15sM36M 24 
+ M16M34M52 + Mj6M35Ma2 + M16M32Mas, (11.70) 


where m;; = E[X; X;]. These characters as shown in Eqs. 11.68, 11.69, and 11.70 were 
fully utilized in the calculations of the following together with the symmetry properties of 
hz (uy, U2) h3( qi, G2, 73) in terms of Uy, U2; 41, 92, 93. 


1. Mean of Y(t), my = E[Y(2)] 


my = | | ho(u4, U2) Ry(u2—u,) duyduz 


ho(uy, u2) | sxx(@) exp{- i(wu; + Wu2)} dw - dujduz 


—oO —c 


ll 
| 
3° 8 
_— 8 


| #@,-0) Sxx(w) dw. (11.71) 


Equation 11.71 is the same expression with Eq. 11.33’ for the quadratic process. 
2. Second order covariance function, cross spectrum 


Ryx(t) = El{¥@)— my} X¢-9)] 


= | hy(r)Rxx(t —r) dr 


+3 | | | h3(qi, 92,93) Rxx(q2— 91) Rxx(t— 93) dqidq2rdq3. (11.72) 


Therefore 


Syx(@) = Hy(@)sxx(w) + 3 | Hx@1-0,.0) Sxx(@1) Sxx(w) dw 


= Sxx(w) [H;(@) +3 | Hato,-01,0 Sxx(@ 1) dw]. (11.73) 


—-o 
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The term syy(w)/syx(w) is no longer equivalent to H,(w), but is modified by some term 
that includes the third order nonlinear frequency response function H3. 


3. Second order autocovariance function, linear spectrum 
Ryy(t) = E[Y(t)— my] [Y(t + T) — my] 
= E[Y(t)- Y(t+1)] —m} 


oOo ow 


= | [ huts phurDe{xt—oXte+1—ryldsidr 


—O —& 
fo 2) 


t {| | | frcuus hay, v2) E[X(t—uy) X(t—u2) X(t+T—-V}) 


ie xX X(t+T—V2)] dujduz dv\dv2 


rn | | | | | | h3(P1, P2, P3)h3(q1, 92, 93) 


x E|X(r— pi) X(¢—p2) X(¢—ps)-Xe+7- qu) Xe+t-q2) Xe+7—gs)| 
dp dp2dp3dq\dq2dq3 
—m>. (11.74) 


We take the Fourier transform, and the linear spectrum is, after manipulation, 


(oe) 


Syy(W = Sxx(w) | Hy\(w) +3 | H:@,03,-03) Sxx(w3) do | * 


—-o 


+2 | | H2(w — 2,02) | * syx(w —@2) Sxx(@2) doz 


{| H3(@ — @2,— 03, 2, W3) | 2 syx(W —@2— 03) Sxx(@3) dwrdw3, 


Fis (11.75) 


_ The first term, the term of syx(w) of this Eq. 11.75 is again modified by the cubic 
response H3, as was Eq. 11.73 that shows syx(w)/sxx(w) is not H;(w) anymore, but is 


modified by the effect of cubic response H3(@1,@2,w3). Bedrosian and Rice®° showed 
this modification clearly for the case when the input was the sum of sinusoidal waves, 
and showed the necessity to include the higher order terms in Voltera expansions. 
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4. Third moment cross covariance and cross bispectrum 
ioe) oe) 


Ryxx(T1, 72) = 2 | | h2(uj, U2) Ryx(t1— 1) Rxx{t2—u2) dujdu2, (11.76) 
therefore SYXX(W 1,2) = 2H2(W1,W2) Sxx(@1) Sxx(W2). (11.77) 


If we use X7(r) instead of Y(t), from Eqs. 11.56 and 11.57 we get 
Ryoyx(T, T2) = 2RxxT1) Rxx(T2) (11.78) 
Sxy2xx(W1,@2) = 2sxx(@1) Sxx(@2) (11.79) 


as we did in the preceding section for Eq. 11.50, inverting the variables from w ,w> into 
W1—-@2,0,+@2, 
2 Syxx (W@1 —@2,@1 + W2) = Syxx(@1, 2). (11.80) 

Therefore, the quadratic frequency response function H> (@; 2) is obtained by 

SYXX(@1,@2)  __S¥xx(@1,2) (11.81) 
25xx(W 1)5xx(@2) — Sx2xx(W1, @2) 
_ SYXX(@1—@2,@1+@2) _ S¥xx(@1—@2,@1 + @2) (11.81’) 

Sxx(@1) Sxx{W2) Sx2xx{@1 —@2,@1 + W2) 


These are the same as Eqs. 11.49, 11.55, 11.58, and 11.59 for the quadratic process and it 
means that the process to calculate the quadratic response H2 (w 1,2) is the same as for 


the quadratic nonlinear process, even for this third order nonlinear process. 


Hx(@1,@2) = 


5. Fourth moment cross correlation and trispectrum 


When Y(t) is expressed by Eq. 11.67, which includes the third order nonlinear for 
the cubic nonlinear process term, the fourth moment cross correlation can be obtained by 
the same kind of manipulation as in the quadratic nonlinear process as 


Ryxxx(t, 72,73) = E[{¥(t)—my} X(t-11) X(t—t) X(t-73)] 
26 | 


Its three—dimensional Fourier transform is called the trispectrum and is 
SYXXx(W 1,2, 03) = 6 H3(@1,W2,@3) Sxx(@1) Sxx(@2) Sxx(w3). —(11.83) 
If we use X°(z) instead of Y(t), then 
Rysyyx(T1, 72,73) = ORxx(T )Rxx(T2)Rxx(Ts) (11.84) 
Sy3 yyy(W1, 2,03) = OSxx(W1)Sxx(W2)Sxx(W3). (11.85) 


_— 8 
_— 8 


h3(q41,92,93,) Rxx(t— 41) Rxx(t2— 92) Rxx(t3 — q3) 


dqidq2zdq3. (11.82) 


8 
8 
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Therefore, the third order nonlinear frequency response function is obtained as 


SYXXX(W 1),@2,(@3) —_ S¥Xxx(@1, @2, 3) (11.86) 
OSxx(W1) Sxx(@ 2)Sxx(@3) Sys xxy(W1,@2, 03) 


H3(@1,@2,@3) = 


6. Application of the trispectrum to ship’s rolling by Dalzell. 


J. F. Dalzell!’ discussed the characters of higher order nonlinear responses in Vol- 
tera expansion, especially the interferences between different order nonlinear responses, 
based on E. Bedrosian and S. O. Rice® and also tried to solve the problems encountered 
on their numerical computations. Besides Dalzell’s trial°>-** on the application of the 
bispectrum analysis to the added resistance of ships advancing in waves (already referred 
to in Section 11.4), he also tried!” an application of trispectrum analysis to the problem of 


nonlinear ship’s rolling, expressing*! the nonlinear roll damping by cube of roll angular 
velocity. 


Dalzell expressed the equation of motion as 


3 
[AilPo}+ B Yoh + cro} = X(2). (11.87) 
j=1 


S 


Then, using the so-called incommensurate frequency technique,” the relations of 
Aj, Bj,C;j = 1 — 3) and frequency response functions H;(w), H2(@,,@2), 
H3(@1,@2,@3) were derived. 


Setting appropriate values for A, to C3, and using the simulated series of wave 
heights X(r) that have Pierson—Moskowitz type spectra, Dalzell synthesized the roll 
response Y(t) as shown in Fig. 11.12, using impulse responses h)(T), h2(t,T2) and 
h3(T), 72,73) obtained through H\(w), H2(@1,@2) and H3(w1,@2,w3) expressed by Aj, B;, 


and Cj(j=1 — 3). 
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Fig. 11.12. Simulated time histories of linear random excitation X(f and nonlinear 
response Y(t), and its components Y4(t), Yo(t), and Y(t) o, = 1.0. 


(From Daizell.17) 


Here, Y;(t), Y2(t), and Y3(t) show the first, second, and third terms of the Voltera 
expansion of Y(t) as Eq. 11.67. 


Figures 11.13 and 11.14 show the first and second order impulse response functions 
h,(t) and ho(t), t2) in the form of the weighting functions 8}. Sit and Fig. 11.15 shows a 
portion of the third order impulse response function h3(t), tz, t3) at the sequence of f3 val- 
ues in the form Set: It is interesting that impulse responses are obtained quite beautifully 
by this analysis. 


Fig. 11.13. Truncated linear discrete kernel g}. 
(From Dalzell.17) 
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Figure 11.16 compares the observed and the predicted second order moment spectra 
and also U;, U2, and U3 correspond to the first, second, and third terms of Eq. 11.75. It is 
rather surprising that the observed value can be almost fully explained theoretically by 
Eq. 11.75. It is also interesting to find in the upper second figure of Fig. 11.16, that the 
linear estimated spectrum (shown by solid line) is pretty largely distorted by the existence 
of cubic frequency response H3 as are shown by Eq. 11.73 and by the first term of Eq. 
11.75, and is modified into U; (w) shown by the dotted line in the same figure. 
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Fig. 11.14. Truncated quadratic discrete kernel g;,. 
(From Dalzell.17) 
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Fig. 11.15. Portions of the truncated cubic discrete kernel g3,. 
(From Dalzell.17) 
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Fig. 11.16. Observed and predicted spectra and the components of spectrum of 
nonlinear response of the simulated system, nominal o, = 1.0. 
(From Dalzell.17) 
7. Expansion to higher order nonlinear process. 
By the same procedure as previously used for quadratic and cubic nonlinear pro- 
cesses, we can go on to higher order nonlinearities. For example, from Brillinger,® 
Syxx. ..x(@1,@2° * *@m) 


pescewinas 0.0. Cuey Qe (11.88) 
m!Sxx(@1) Sxx(@2)* + - Sxx(@m) 


Ay (@1,@2,° - *®m) = 
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CHAPTER 12 
PROBABILISTIC CHARACTERS OF NONLINEAR RESPONSE PROCESS 


12.1 INTRODUCTION 


The stochastic or random process can be expressed as a function of tme and also as 
a probabilisitc measure x(t,s). Until Chapter 11, however, we have discussed the charac- 
ters of the process mostly in the time aspect and, based on the ergodicity, dealt with the 
correlation functions and spectra and their statistical characteristics. Especially in 
Chapters 9, 10, and 11, the effects of nonlinearities on these characteristics were studied. 


In this chapter, the probability distributions that characterize the response process 
will be discussed briefly for reference. The main topic in Part II is the nonlinearity of the 
process. However, general considerations, including the linear cases, will be discussed 
first. 


Generally, the probabilistic character of the random process X(t) can be defined 
completely by the series of probability distribution density functions as 


Pi(x1,t;)dx : the probability that X(t) is in the range x) — x; + dx 


at tme 14; 


P2(%1,t1; X2,t2) : the probability that X(t) is in the range x) — x1 + ax 
at time 7 


and in x2 — X2+ dx at time 19; 


P3(X1, t1; X2, 12; X3, 3) : the probability that X(t) is in the range xj — x) + dx) 
at time 7), 
in X2 — X2+dx> at time fp, 


and in x3 — x3+dx3 at time f3. 


In the same way p, is defined for n=4,5... . 
Here i. p, 20 
ii. Pp 1S Symmetrical for x), tf); X2, t23X3, 13; ... 


iii. the marginal probability density function p,, is 


Pm = | see | p. dXm+1dXm+2 016 6 Gx (12.1) 


As was mentioned in Section 2.1, when these probability density functions p, stay 
the same when the time 1; is replaced by 1,+T7 (T: arbitrary), this process is called station- 
ary. Then pj(xj, t}) is independent of time, and p> depends only on the time difference 
t7-t). Here the concept of the Markov process that is used to classify the random process 
probability is first introduced. 


From the characteristics of these distribution functions, the processes are sometimes 
classified into three groups: (1) pure random processes, (2) Markov processes, and (3) 
general processes. 


For a pure random process, since the value x; at time 7 is independent of any values 
of x(t) at any other times 4-1, f;-2, . . . t2, 41, this process can be defined completely by 


P1(%,t;), because any higher order joint probability distribution function is 


n 
Pn(X1, 13 X2,t2 - - - Xn tn) = _ PGi, ti) . (12.2) 
I= 


Accordingly, if only p;(%;,t;) is given, all other joint distribution functions can be derived 
easily. 
12.2 MARKOV PROCESS 

When the process is not purely random, but when in addition to p like p,(x;,t;) and 
PiQi-1,ti-1), if p2(%-1, tj-1; xj, tj) 1s necessary to express the probability characters of the 
process, the process is called a Markov (or Markovian) (linear) process. The condition 
that the probability distribution density functions p(x;-1, t-1) and p2(%}-1, t}-1; %j, tj) are 
given is the same as the condition that the conditional probability distribution density 
functions p-,(x;, tj — t;-11xj-1) are known. Because the conditional probability density func- 
tion Po,(%;, tlxj-t}-1) is the probability distribution density function of this process that x 
is in the range x; — x;+ dx; at time ¢;, under the condition that the value of this process 
was in the range of xj) — xj-1 + dx; at time 4,4, thatis (¢;— 7, ) prior to 7, and is 
expressed by 
P2(X;, tj, Xj-1,t;-1) 


(12.3) 
PiQr 1) 


Dey (Xj, til Xj-1, ti-1) = Po,(Xjp tj — Hailxp) = 


Here, from the characteristics of the probability distribution density function, 


1. PcAXjp tlx;-1) =>0 
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il. | re tx;-1)dx; =1 


iii. pi(%j,%) = [ press tedpatsnt— tabs. 


Generally, the Markov process is defined as follows, using the conditional 
probability when the probability, that X(t) is in the range x; — x;+ dx; at 4, under 
the condition that 


X(t) is oF] Ps +dx;1, at 1, 


xj-2 X;-2 + dx;_2, at tj2, 


X2 — 12+ dx, at tp, 


Xx; — X,+ dx, at ty, 
is determined by the conditional probability that X(r) is x; — x; + dx;, under the condi- 
tion that at only one preceding time step f = tj, x(t) 18 x-1, then this process is 
called a Markov process. Namely, if 


JOC EPSP IGCPOLIROR oo 5 8 BADUBES HH) 


SY CREPE) (12.4) 


this process X(t) is a Markov process. 
From Eg. 12.3 and Eq. 12.4, generally 


DACE pEPIE ETS 6 a 6 LPHUBES UN) Je Cry be sil) 
oS JO MCP IRCE ORE SP 6 0 6 AM PRAM UL (12.5) 


Therefore, by the same relation, 


Pr(&n, tn Xn-1,ln-1; > - - X2, 123X1, ty) 


= Do (Xn, Clete tn-1) Pe{Xn-1, tell; tn-2) sos 


X Pe,(X2, talx1, t1)p1@1, ¢1). (12.6) 
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From this expression, the term Markov chain is used. The conditional probability 
function is also called the transitional probability function. 


Also 
c 
D2Gns tn; Xn-2, =D) = | D3(Xn, tn3 Xn—-15 tn-15 55 Di_5) aXp-1 
— 
and 
P2(Xn; tn Xn-2, tn-2) = Pc,Xn; tnlXn—2, tn2) P(%n-2; tr_2) 
P3(Xn, tn; Xn-15 ln-13 Xn-2, t=?) = PDc(Xns tn; Xn-1, lS Diino) 
x DXn-2; tn-2) 
therefore 
ce 
Pe{Xn» TalXee, tn2) = | DeGonstni knelt trenlcnls ty-2)AXn-1 (12.7) 
—0 


this Eq. 12.7 is called a Chapman—Kolomogorov equation or Smoluchowski equation for 
the transitional distribution of the Markov process. It implies that to go from x,_2 at time 


tn_2 tO x, at time ¢,, the path x,_; at time f,,_; is not important. We can take any path to 
go to x,. This characteristic is used to derive the Fokker—Planck equations, shown in the 
next section. 


For example, the AR model of the first order AR(1) as was treated in Chapter 5 is 
X(t) -—axX(t—1) = €(0), (12.8) 


where a is a constant, €(t) is a pure random process (a Markovian linear process), and so 
X(t) is statistically determined by the value X(t-1), that is, the value of X(t) at one preced- 
ing time step. Accordingly, X(r) is a Markovian process. 


12.3 GENERAL PROCESS 


If we need the values of a process not only at one preceding time point f,_;, but at 
two or more preceding time points, i.e., not only p; and p2, but also higher order joint 
probability functions p3, p4, .. . the process is called a general process. However, among 
general processes X(t), some can be inverted into vector Markov processes, combining 
with some other variables and introducing the concept of the state space (Markovian). 


Putting aside the concise theory, for example for the second order autoregressive 
model AR(2) as 
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X(t) — a X(t — 1) —a2X(t— 2) = (1), (12.9) 


we can introduce the vector process X(t) with two elements X), X2 as 


X(t) = X(t) 


(12.10) 
X1(t) = a2X(t— 1)(= a2X2(t- 1). 
Then Eq. 12.9 can be written 
X2(t) 1 a, | | X2¢-D 1 
X() X(t-1) 
Here setting 
Xy(t) 0 a2 0 
= X(t), =a, €(t) = €(f), 
X0(t) (t) aie By \e (t) = €(¢) 
Eq. 12.11 becomes 
X(t) = aX(t-1)+€(0). (12.12) 


This transformation shows that AR(2) can be inverted into a Markovian linear process or 
a vector process X(t) = [Xj(t), X2(t)]’. 


By the same procedure, the autoregressive process of order n, AR(n), or more 
generally the autoregressive moving average model ARMA (an, m), can be shown to be 
invertible into a Markovian process, introducing the matrix of inverting functions. 


The characteristics of a Markovian process contribute to the derivation of distribu- 
tion functions, applying the Fokker—Planck equation, and also for the application of the 
Kalman filters. 


12.4 THE FOKKER-PLANCK EQUATION 


To show the behavior of the transitional probability density function p, of a stochas- 
tic process as a Markov process, formally a linear second order partial differential 
equation called the Fokker—Planck equation is used. This equation was developed by 
Fokker (1914) and Planck (1917), to indicate the Brownian motion of molecules, and it 
has also been utilized as the equation of motion to show the behavior of the transitional 
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probability density function of a stochastic process.°3°4 This equation is, however, not 
solved analytically in general, except for a few special cases, so special considerations are 
usually demanded in using it. 


Here the conditional or transitional probability density function p-,(x, Atly) is 


expressed merely as p., y being the value of process X(t) at a preceding time Ar. The 
derivation of this equation is given by Caughey.* 
The Fokker—Planck equation for this transitional probability function is in the form 


apc d las 
— = | —-— p(x) + —— D(x) I p.. 12.13 
at ax ep 2 ax ©) |e Seay 


Here D(x) and Dx)>0 are called the drift coefficient and diffusion coefficient, 
respectively, and are related to the first and second moments of this distribution as 


DOC) = lim Hea [ csp ayyay (12.14) 
Aro At 
1 

D?x) = lim — | (y —x)*pe(x, Atly)dy. (12.15) 
At > 0 At 


Generally these are functions of time t. 

As was stated before, when the general process is inverted into a vector Markov 
process X(r) of N dimensions as X(t) = [X;(t), X2(t) . . . Xy(t)]’, the Fokker—Planck 
equation is more generally in the form 


N N 2 
ezene - Sy 2 Mx 44 » ke) Pe (12.16) 


ot sa OXi 2 721 9x10; v 
where 
(Dyce 1 N 
D; (x)= lim — (yi-x) p(x, Atl y)T1 dy; (12.17) 
Aro At fall 
eR uen a Nv 
Dj (x)= lim — (yi-xi)(Vj—X)Pc( x, Atl y)TI dy;}. (12.18) 
Ar>o At i=1 


—-o 


x, y are the values of process X(r). 
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Solving Eqs. 12.13 or 12.16 gives the transitional probability functions. Since these 
equations are solved analytically only for some restricted types, several methods, such as 
a simulation method, transformation into a Schradinger equation, numerical integration, 
potential method, and so on, are used to obtain the solution. 


When, by the elapse of time, the transitional probability function p(x, tly) tends to 
p(x) as its limit and becomes stationary, independent of time and its initial value, then in 


a 
the Fokker—Planck Eqs. 12.13 and 12.16, setting > ©, = £=0 


’ 


2 ae] 2 Oras es 
Fl 


OX;0X; ja]. (OX: 


where p(x) obtained as the solution of this equation is an N—dimensional joint distribution 
related to the N—dimensional state space for this vector process X(t). 

Under some considerations and restrictions, these methods can be expanded to deal 
with nonlinear processes. 

For example, following Caughey,*? consider a nonlinear oscillation with nonlinear 
restoration under Gaussian (mean = 0) excitation as 


X(t) + BX(t) + FIX} = Ko) (12.20) 
E|fit)] = 0 (12.21) 

Wi 
Elft:) Atz)] = 5 th sf) (12.22) 


here Wo/2 shows the white spectrum of the excitation, and 6 is the Dirac’s delta 
function. From the transform 


Xi) =X) 
‘ (12.23) 
X(t) = X() 
and introducing the vector process 
X(t) = [X1(0), Xo(0))' = XO, XN, (12.24) 


from Eqs. 12.20 and 12.23 
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X4(t) = X20) 


(12.25) 
X(t) = — BX2(t) — F[X1(0)} + Ad). 


Coefficients Dx), and De (x) for the Fokker—Planck equations are, when we 
insert Eq. 12.23 into Eqs. 12.17 and 12.18, 


Dp = lim E[AX,] Bate 3 
At—~0O At 
(12.26) 
DW = he E[AX] 
Msg 28 
E[AX? 
DY? = lim alieg 
At—~0O At 
E[AX, AX 
Diz = Dy = lim . Ag (12.27) 
t= 


Substituting Eq. 12.25 for E[AX>] and E[AX3] and using t as a dummy variable in 
t gives 


t+At 


E| |-BX2(t)—F(Xi())}Ar+ J fe)de 


t 


Dew = lim 
At—~0O At 
= — BX» — F(X1) (12.28) 


332 


t+Ar 2 
E| } {|-BX2(t)— F(X1(0))|Ar+ | fAidddr 
(2) d 
Dy = lim 
Ar—>0 At 
t+Ar 
= lim E}{-BX2—F(X))|’Ar+ 2{/- 8X2 - F(X))} | frat 
At—>0 oie Pe. 
SS & Ne 9 os ae 
t+Ar t+Ar 
+ lim E it | | Ft p)ft2)at dt 
At—>0O At 
t t 
Wo 
Ss 12.29 
5 ( ) 


Now, the process is stationary, so with Eq. 12.19, the Fokker—Planck equation becomes 


Wo ap a a 
eee aes (np) +5 [(BxX2+ F&XyIp] (12.30) 


This is called the stationary case for Kramer’s equation and has been solved by several 
scholars. Following Caughey—Wu, the solution of Eq. 12.30 is 


x) 


48 |x 
D(X1, X2) = p(x, x) =C exp -- ee | Feat k (12.31) 
0 


Here C is the coefficient for normalization and € is a dummy variable for x;. Equation 
12.31 is in the form of 


E 
p(x, x) =C exp, —48 -—}, (12.32) 


Wo 


where E is the total energy per unit mass of this system 
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Boi 
Boe | F(6)dé. (12.33) 
0 
From Eq. 12.31 
p(x,x) = p(x)p(x), (12.34) 
and 
p(x) = of ee (12.35) 
Wo 
p(x) =C ex mae | F(6)dé |, (12.36) 
Wo 


0 


we find the probability distribution density function of the velocity x is a Gaussian. 


More concretely, for a nonlinear spring system, F{X(z)} in Eq. 12.20 is 


FIX,()} = FIX(o} = wf X . ea(X(0)] (12.37) 
and the equation of oscillation Eq. 12.20 is 

K(0) + BX(0) + 3 Xow) + eg{X(}] = Ao. (12.38) 
Here for hard spring type, € > 0, and € is on the order of x(t)/g{x(¢)}, g{x(2)} = 
—g{—x(z)}, and as Lx(z)|>0, x(t)g{x(t)| > 0. Now we define wo as the 


undamped (when # = 0) natural frequency of oscillation. 


The mean square value of this oscillation is 
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——, 


= | x°p(x, x)dxdx 


—c 


E\X*() 


= | Ee + eg(x)} - seta) [pena (12.39) 
From Eq. 12.36 and Eq. 12.37 


x x 


il i |) ae 
PC Grexp -,| Fea ='G exp oe a Ge) 
0 


G(x) = | g(€)de. 
0 


For the linear system, € = 0; therefore from Eq. 12.39, 


loo) 


E[X*(t)] = of - | xeg(x) p(x)dx 


—co 

= ofx-€ E[x(x) a(x]. (12.40) 

For a hard spring system the mean square of the displacement is always smaller than 

it is for the corresponding linear system. 
For the so-called Duffing type system, g{X(t)} = x0), and the equation of motion 


is 


XO) + BX) + w6 {X() + eXx*(0)| = fit). (12.41) 
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Therefore from Eq. 12.40 
E[X°@)| = of -€ LX’). (12.42) 


For the Gaussian process E[X“(t)] = 30,*. Therefore in this case, the order is also in 
E[X*(1)] = 02 —3e0%. (12.43) 


It is interesting to find that this approximation is the same type as Eq. 10.17 
obtained through the equivalent linearization method and Eq. 10.50 obtained through 
the perturbation method. 


12.5 PROBABILITY CHARACTERISTICS OF AMPLITUDES, 
MAXIMA AND MINIMA 


S. O. Rice® has shown that, when the joint probability density function of x and 


x, the values of a stochastic process X(t) at time t, and its derivative (d/dt)X(f), i.e., 


p(x,X), are known, the frequency of occurrence of any threshold value crossing or zero 
crossing is easily derived. 


For example, the expected value of the frequency of occurrence of the process X(t) 
for the upward crossing of a threshold of level a, E[N,(a)], is 


E[N,(a)] = | xp(a, x)dx. (12.44) 
0 


If x(t) and x(t) are independent, p(x, x) = p(x) - p(x), therefore 


EIN+(@)] _ [p@)]x=a 


= (12.45) 
FIN.()] [p@)].=0 


When the power spectrum is narrow banded, the amplitude of the sample process 
varies slowly, just like the envelope of sine waves, and as the result, the process is 
assumed to have an extreme positive value a, at each cycle of this sine wave. 

Then the probability distribution is 


_ E[N.(@)] 


ee 2) TENA 


=1-P(@p, S a). (12.46) 


Therefore, the probability density distribution is 
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dP 1 dQ(E[N.(a)]) 
a) = = Ba ee Sis ee eee 12.47 
Pp (a) da E[N,.(0)] da ‘ 


When X(t) and X(t) are independent, 


dp(x) 
dx |x=a 


(12.48) 
p(0) 


Py (a) = 


Expanding this kind of relation, Longuet-Higgins, * and Cartwright and Longuet— 
Higgins,!> developed the well-known results for the distribution function of extreme 
values of a general process with various bandwidths. 


The probability distribution density function for extreme values of a process X(t), 
or &),&>, .. . normalized by the standard variation of the original process 0,, as 


E/mo = &/o, = was derived as, 


raed? 

Sa r 

Ge Sees | cea (12.49) 
Dp Sire €e ~ +(1—€%~)*Ce e u : 


~—  —& 


as shown in Fig. 12.1, where € is the so—called bandwidth parameter of the spectrum 
<= moma—ms) 
moms, 


(12.50) 


> 


m,, being the moment of spectrum 


Mm, = | wrsoyde. (25D) 


—-o 


When ¢€ = 0, as for a narrow banded spectrum, p(¢) becomes a Rayleigh distribution as 
is well known, 


ase, 
po) = ie c= 0 (12.52) 


or 


331 


or 


pé)=sa2 ; 3 (12.52’) 
0 <0 


At the other extreme, when € = 1, as when ripples are superimposed on slowly varying 
waves, 


(12.53) 


1 am) : 
p) = oe e “* = p(x), (23539) 


namely p(€) becomes Gaussian and it is the same as the Gaussian distribution that 
governs the original process x(t). 

When € is between 0 and 1, p(€) is given by Eq. 12.49 and is between the 
Rayleigh and Gaussian distributions, as shown in Fig. 12.1, and is now popular for us. 


2) 
o 


p(c) 


2 © 
how 


oOo 


-3 2 -1 0 Hv aeionve 3 


Fig. 12.1. Probability distribution density function of extremes. 
(From Cartwright, Longuet—Higgins.'%) 


From this distribution, the expected highest value of 1/n and the expected highest 
values of N independent samples have been derived as functions of €. The derivation will 
not be referred here, as they are well known. 
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Equations 12.46 and 12.47 can be applied to a nonlinear process as long as the 
process is narrow banded. Crandall®* analyzed a hardened spring oscillation system, the 
same as the one analyzed by Caughey* utilizing the Fokker—Planck equation, as was 
shown in Section 12.4. Equation of motion is by Eq. 12.20, 


X(t) + BX(t) + F{X(2)| = fir). 


He obtained the average period t(a), the probability distribution function of the peak val- 
ues pp(a), and the probability distribution function of the envelope p,(a) as functions of 
the amplitude a by introducing the potential energy per unit mass 


x 
V(x) = | F(6)dé. (12.54) 
0 
The solution of the Fokker—Planck equation, Eq. 12.31 (when f(x) is Gaussian) is 
Tea | exe 
p(x,x) =C exp A) ap Vix)? }, (12.55) 
of| 2 


where C is a normalizing constant that makes 
ioe] foo) 
| | Deak) adn = i 
—-Om —O 


The results are, as for the envelope process a(t), 


V(a) = 2 + V(x), (12.56) 
a [2[Via)-Vix)]f2 
Pia. <a)=4 | dx | p(x, x) dx (12.57) 
0 0 
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dP 1 
pea) 2 ap le [2 V(a) - veel (x.[2 ,[2{ V(a)- vealf) ae C28) 
0 


Through manipulation, the following were derived: 


pea) =C-t(a)- F(a) exp <4 = pp(avg' T(a) (12.59) 


xe 


and also, 


V 
pp(a) = F(a)-0;? exp 4- ~ ; (12.60) 


x 


where 7% is the expected number of threshold crossings per unit time of the level x = 0 
with positive slope, and can be derived as v4 = Co?; t(a) is the average period as a 
function of amplitude a. 
Crandall showed the general solution for the following two nonlinear cases: 
i) fora hard spring Duffing type system, 


FIK()) = wg {X(e) + x20}, (12.61) 
ii) fora set-up spring system, as in Fig. 12.2, 


F(X) = 0g {X() +e sgn XO}, (12.62) 
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Fig. 12.2. Displacement—force relation of set-up spring. 


as are shown in Figs. 12.3 and 12.4. Hereo, is the standard variation of the linear system 


a 
when € = 0. These figures show that the probability distribution of extreme p, mi 


x 


varies considerably with the extent of nonlinearity, expressed by the value of e€. 


Fig. 12.3. Probability distribution density of the extreme (peak) 
values of Duffing type oscillator. 


(From Crandall.®4) 
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Fig. 12.4. Probability distribution density of the extreme (peak) 
values of oscillator with set—up springs. 
(From Crandall.°4) 


12.6 APPLICATION OF THE FOKKER—-PLANCK EQUATION FOR 
THE ANALYSIS OF SEAKEEPING DATA 


As has already been mentioned, the Fokker—Planck equations have been solved only 
for a restricted number of cases, and this has made the applicability of this equation rather 
difficult in many engineering fields. For example, this method was applied to the nonlin- 
ear system with nonlinear restoring terms and with Gaussian white excitation, as shown 
in the preceding section, but the method has been considered inapplicable to the system 
with nonlinear damping or with colored noise excitation. A few efforts have been made to 
overcome these difficulties, for example by J. B. Roberts.°? His work, *! especially in 
the analysis of nonlinear seakeeping data, will be summarized briefly. 


12.6.1 Nonlinear Analysis of Slow Drift Oscillation of 
Moored Vessels in Random Seas 


Utilizing the known characteristics of the Fokker—Planck equation, Roberts 
analyzed the statistical behavior of the nonlinear slow drift motion of moored vessels. 


Assuming that the waves 7(t) are narrow banded with band width parameter e, and that 
the drifting force D(t) can be regarded as proportional to the square of the wave height, he 
modified the expression of wave height 7(r) and drifting force D(t) to 


n(t) = H(t) cos [wot + o(t)] = hA(t) cos [Wot + o(2)] 
A al {a2 i: on} | cos [wot +a(2)] 


= h[a(t) cos wot — b(t) sin wot], (12.63) 
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here Wo is the peak frequency of the narrow banded spectrum of the wave, and 
D(t) = a[H?(1)] = a'[a*(t) + b*(1)] = D+ k'E(0), (12.64) 


where H(t), hA(t) are the amplitudes of the envelope; a(t) and b(t) are components of 
A(t); a(t) is the phase lag; D is the mean drifting force; and &(r) is the white noise, as will 
be assumed later. 

The equation of swaying motion of the vessel was expressed as 


(M + m)x + F(x,x) = D(t) (12.65) 


or 


X + flx, x) = Bla2(t) + b7(0)] (12.66) 


where (M + m) is the virtual mass of the vessel. Then with a(t), b(t) as the output of white 
noise &(t) through linear filters, the equation of motion was modified to 


z+fiz )= E(t). (12.67) 


The term z is a vector Markov process with four elements [x,x,a,b], and the Fok- 
ker—Planck equation of this four—element variable z(t) was then derived. Then since 
A(t) can be generated as the output of a nonlinear first-order system with a white noise 
input &(t), the process was inverted into a three-element vector Markov pro- 
cess, y(t)[x(t), x(t), A(t)], and the Fokker—Planck equation for this process was also 
derived. 


However, the solution of the multidimensional Fokker—Planck equation has a num- 
ber of difficulties, so Roberts advanced the approximation further, and expressed the 
drifting force as 


DH=D+k'E(s), (12.68) 
the sum of the mean drifting force D, that can be approximated as a’e, plus a white noise. 


Under these approximations, the equation of swaying motion is 


X+ g(x, x) = d+ k&(t), (12.69) 
where d= 2 = Gis = Be, 
(M+m) (M+m) 
k= ia : 
(M +m) 
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The two-dimensional Fokker—Planck equation was derived as 


ap a a bP ap 
eS Mee Gp Bie Oy EE (12.70) 
a am az’ Daze 


where Z; =x, Z,; =x, and pis the transitional probability density function. 


The stationary solution of p is obtained by (dp/dt) = 0. As g(x, x) in Eq. 12.69, in 
its general form 


2(x, x) = yF(x, x) + G(x), (12.71) 


is used. 


Then the joint probability density function of swaying motion p(x, x) is the 
solution of Eq. 12.70, when (dp/dt) = 0, 


pO =i as|-Eo ae U(x) — dx (2772) 
yD, 
where 
U(x) = | G(A)dh (12.73) 
0 


is the potential energy for restoring forces, C is a normalization constant, 


and 
Q(y) = [2a (12.74) 
0 
Here 
1 
= J2v— 12.75 
Biv) TIA) | F(x) ¥2(v — U(x) dx ( ) 
R 
C(v) = as | vv—U(x) dx (12.76) 
R 
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A(v) = = | ———. (12.77) 
yy y 


The integration range R is such that U(x) < v. Especially for Duffing type oscillation the 
equation of motion, Eq. 12.69, is expressed by 


eee oo. (12.78) 
On, Wn 


Here ¢ is replaced by nondimensional timet,t = @,t, w, being the undamped natural 
frequency in the linear case (A = 0), and the nondimensionalized equation is 


X' +2 Cx’ +x" +A* x3 =d* +2 CEQ), (12.79) 
where the differentiation is with respect tot and 


t(1 + 4ex2) |? € 


= | ———_|_ ,e* = —. 


1/2 
mcs, jee e pa Zes |S. 
Ox o; k \o, e* Op, 


The joint density distribution for p(x’, x’) and the probability density distribution 
function p(x’) is, from Eq. 12.72, 


foo Na G —4—+—+ (7) 85d (12.80 
p(x, x’) exp aS i % ) 
and 
* 2 12 R* 14 
pe) = | po x')dx' = /2a C exp |- a 2 ae \\ | (12.81) 


—c 


Here C is a coefficient for normalization, 
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pee axidca—Ne 


_—— 8 


Roberts calculated numerically, for a Duffing type system, with various €, A*, and 
€* and compared them with the simulated results. Figure 12.5 shows an example of his 
results. He compared p(x) calculated by Eq. 12.81 under the white noise approximation 
with the result obtained by simulated data and found that the agreement is good and quite 


different from the Gaussian distribution for A* = 0. Once p(x) of the displacement x(t) is 
determined, p(T), the probability distribution of tension on the mooring, can be derived as 


8 


(x) 
pe) aes, (12.82) 
aw 
dx 
where the tension is expressed by 
T =w(2). (12.83) 


fe \ » = 0 
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Fig. 12.5. Probability density function for displacement response; 
A* = 0.5,€ = 0.05, e* = 1.0, 0 digital simulation estimate. 
(From Roberts.©°) 
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12.6.2 Stationary Response of Oscillations with Nonlinear 
Damping to Random Excitation 


As was mentioned at the beginning of this section, the Fokker—Planck equation has 
been solved only for restricted cases, such as nonlinear systems with nonlinear restoring 
forces or systems under white noise excitation. Roberts*? removed these restrictions and 
studied the behavior of the Fokker—Planck equation for a system with nonlinear damping. 
For a nonlinear system with nonlinear damping, the equation of motion is 


X + BF(x, x) + G(x) = n(t), (12.84) 


where f is small, F is an odd function of % (odd as was used by Dalzell®! is more 
convenient to manipulate), and n(t) is white noise, G(x) being the restoring term. 


The two—dimensional Fokker—Planck equation for 
P(x, x) = p(x, yl%o, Yo; 2) 


is the same as Eq. 12.30, for Eq. 12.20 in the form, 


op op 0 Jf ap 
a SS SJ SS F ———- 12. 
an ae + ay [6 (x,y) + G(x)|p] + BER (12.85) 


and is difficult to solve for general F(x, y), so Roberts inverted this two—dimensional 
Fokker—Planck equation into a one-dimensional Fokker—Planck equation, introducing a 
physical variable called the energy envelope V(t) 


V(t) = +2 + U(x), (12.86) 


x 
therefore x = /2V(t)— U(d) where U(x) = | G(&)dé. 
0 
The one-dimensional Fokker—Planck equation for p(v) = P(v\vo; t) is in the form 


ap a ji InP 
neh Syne Biv) -— ee 5 
a ey (Oa iP aan (12.87) 


W, 
where / is the strength of excitation as ] = oa in Eq. 12.22 or Eq. 12.29, and 


Biv), C(v) areas in Eqs. 12.75—12.77 
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1 
Biv) == | Fixy2(v- dx 
O 2m | {x /2@-0e| 
R 
1 
= ¥(v— U(x) dx 
i fa ann 
R 
A(v) |= 
2) v-U@) 
R 


The integration range R is such that U(x) < v. The general stationary solution is given as 


Pv) = lim p(vlvo; 1) = KAW) of (2 2o)} (12.88) 


where 
Vv 


_ {2© 
O(v) = | C® (12.89) 


0 


and k is a normalizing constant. Then, from the relation of v, x, y(= x) and reverting to 
the original x,y variables, 


1 
p(x, y) = Jim p(, ylxo, yo; t) = C oo (# Jo a + U(x) } (12.90) 


When the equation of motion is expressed as 


X + Wowox(1 + elx!") +6 x = n(t), (12.91) 


NOW Wo being the natural frequency of the linear (€ = 0) system, Eq. 12.90 becomes 


n+2 


1 
P(x, y) = C polx, y) exp4 — Gn€* ate +y’) 


where 
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1 1 
e*= ewfoe, x,y)=— —=( 4+y? (12.92) 
090, Polx,y) ee? 7 | y’) 


x(t) x(t) if 3 
OSS a i aaa 0o= —lo-w6 ’ 
00 W000 4 


Go is the standard deviation of response when € = 0,€ * = €/wo is the nondimensional 
nonlinearity parameter, and po(x,y) is the result whene* = 0 (C = 1 when €* =0). 
From this result, the statistical moments of the response can be calculated. For 
example, Roberts obtained 
2 
(eo) 1 
o = 26|| G8 e? (1—erf 6)| -6| (12.93) 
i) 


u 1 
7 (3e*)!/2 . 


He calculated the equivalent linear damping, as discussed in Section 10.1, using the 
general expression Eq. 12.91 and calculated 


o* (J1+12e*-1) 
— = —————-_ (12.94) 
(oF) 6e€* 


for n = 2 in Eq. 12.91, which is a closer approximation of o* than Eg. 10.17. He also 
calculated the corresponding perturbation solution as 


2 
Ae oeiste (12.95) 
0 


This is slightly different from Eq. 10.50, which was shown for smalle. All three results 
are compared in Fig. 12.6. Robert calls his method, which introduced the energy envelope 
V(t) as Eq. 12.86, the Markov envelope theory, and indicated in Fig. 12.6 as ME theory. 
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Fig. 12.6. Variation of the mean square of the response with e*. 
Simulation results: * > = 0.05, + = 0.50. 
(From Roberts.°9) 


The equivalent linearization theory is abbreviated as EL theory. In this figure, only the 
perturbation theory appears different from the rest, but here we have to remember that in 
the perturbation method this is the first order approximation and we can improve the ap- 
proximation by increasing the order of the approximation, as indicated in the discussion 
in Section 10.2.2. 


12.6.3 Nonlinear Oscillation in Nonwhite Excitation 


Roberts®! further extended the scope of applicability of this method and solved for 
nonlinear rolling in nonwhite excitation. The equation of motion is now 


Ip + BC) + Kb) = M(t) (12.96) 


or 


¢ + BF(¢) + G@) = x(t) (12.96’) 


where x(t) is a nonwhite excitation, and can have a colored spectrum. 
Again, he adopted the total energy envelope as Eq. 12.86 
62 
v=£+u@) 


where 


¢ 
up) = | Ges 
0 
and considered that this was slowly varying. 
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He set 


ro) 
£ =_vsinto @ =—-/2Visin@ 


a (12.97) 


U(d) = Vcos?@ U(p)? = V2 cos 6 


introducing the phase 6 as shown in Fig. 12.7. After manipulation of the relations, the 
equation of motion, Eq. 12.96’, was inverted into the equation of V and@. When the 
phase process 6 was modified into a process A(t), and the joint process Z = (V, A) was 
made to converge into a Markovian process, the transitional distribution function 
p(Z\Zo; t) was found to be governed by a Fokker—Planck equation of second order. The 
Stationary solution of the Fokker—Planck equation p(Z) was obtained, and from this 
expression V and A were found to be uncoupled, and p(V) was calculated. With the 
relations of V, @ and@, p(V) was modified into p(¢,¢). 


Ww OD? 
U,V vs.0,6 


Fig. 12.7. U, Vvs. 6,9. 
(From Roberts.®") 


For nondimensionalized, nonlinear rolling expressed as 
d+abp+ bolod+¢—-? = x(0), (12.98) 


Roberts calculated the probability density function of the nondimensionalized amplitude 
of rolling A by the ME theory and compared it with the Rayleigh distribution obtained by 
the linear theory as in Fig. 12.8. He®!-®6 also compared his results with the nonlinear sim- 
ulated data obtained by J.F. Dalzell®° to show the validity of his method. Examples are 
shown in Figs. 12.9 through 12.11 for a variety of damping coefficients a and b, where 

Q =a,/Wo, @, is the peak frequency of the excitation, andg,, is standard deviation 

of the input process x(t). Process 3 in the figures is a wide banded excitation for this 
example. He showed many other results of comparison for other types of excitation with 
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different bandwidths, named Process 1 and Process 2, used in the simulation by JF. 
Dalzell.° The deviation from the Rayleigh distribution in Figs. 12.8 to 12.11, and the 
relation of the standard deviation of roll op to that of input o,, in Fig. 12.12 shows the 
extent of nonlinearity. The applicability of the method was also discussed. 
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Fig. 12.8. Probability density function for amplitude A: 
a=0, b=1, oy = 0.036, 2 = 0.90, 
Process 3. (From Roberts.®") 


DALZELL’S SIMULATION 
ESTIMATES 
a=0 
i bel 


0 0.1 020.3 04 0.5 0.6 p 
Fig. 12.9. Cumulative probability distribution for amplitude A: 


a=0, b=1, oy = 0.036, Q = 0.90, 
Process 3. (From Roberts.®") 
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Fig. 12.10. Cumulative probability distribution for amplitude A: 
a= 0.01, o, = 0.036, Q2 = 0.90, b = 0.1 and 1.0; 
Process 3. (From Roberts.®1) 
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Fig. 12.11. Cumulative probability distribution for amplitude A: 
a = 0.03, o, = 0.036, 2 = 0.90, b = 0.1 and 3.0; 
Process 3. (From Roberts.®") 
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Fig. 12.12. Variation of standard deviation of roll og with standard deviation of wave 
input o,,: @ = 0.01; Q = 0.90; b = 0.1, 0.3, and 1.0; Process 3. 
(From Roberts.°") 


12.7 PROBABILITY DENSITY FUNCTIONS OF AMPLITUDES, EXTREME 
VALUES IN RELATION WITH THE FUNCTIONAL POLYNOMIALS 


12.7.1 Narrow-Banded Case 


As was mentioned in Section 11.1, when a nonlinear response z(t) was expanded by 


the Voltera expansions (or functional expansions) 
Cc foo) 


z(t) = y. i 5 | Pn, 2) 2 - key) XE — 1X G— 2) x 


n=] —-0 
—S——— 


n 
5 USE) GA G1 oo oo Uy (222) 


the terms for n>2 can be considered as the modifying terms of the Taylor expansion of 
this process around its linear term for n=1. If we take until n=2, and using small €, some- 


times the response z(t) is expressed as 


z(t) = | gi(t) x(t—T) at +€ | | 22(T1, T2)xX(t —T1)xX(t—T2) dtydTz. (12.100) 


—c _ 


The derivatives in terms of time is 


co 


2(t) = | g(t) xX(t—T) att 2 | 


-o —_ 


82(T1,T2)X(t—T1)x(t—T2) dtydt2, (12.101) 


_—— 8 


8 
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thinking of the symmetry character of 92(t),T2) = 22(T2,7)). On the basis of this expres- 
sion T. Vinje®’ formulated a general method for getting the probability distribution 
function of the extreme values, under the assumption that the response z(t) is narrow 
banded and the input x(r) is Gaussian with variance o*. His method is also based on the 
assumption that the probability distribution function of extreme values can be obtained by 
the joint distribution function p(z, z) only. 

Now, we start from the statistical moment generating function to get p(z), 22) 
[here z) = z(t), 22 = 2(t) z], that is, from the double Fourier transform of p(zj, 22), 


(01, 62) = Elexpli(@1z1 + 6222)}). 


= | | exp(i6 Zz) + 10222)p(z1, 22)dzdz2. (12.102) 


Then expanding exp (i612; + 10222) into a series gives 
oo 


Zi 25 
exp (16121 + 1022) =1+ >. = (i6)"(62)", (12.103) 
nO min. 


assuming m,n are positive integers whose sum is greater than zero. 
Inserting Eq. 12.103 into Eq. 12.102 gives 


(01,02) =1+>, ao (iO 1)"(i02)". (12.104) 
Here 
Emn = E[z7, 25] = | | 2725 P(2, 22)dz dz. (12.105) 


From the Fourier inverse transform of Eq. 12.102, 


co 


| | (61,02) exp{-i( 121 + 8222)}d01d82. (12.106) 


—o —0 


1 
P(21, 22) = Ome 


Putting Eq. 12.104 in Eq. 12.106 
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Lmn 


i (10 :)"(i02)" exp|— i( 12) + 0222)|d0 1dO>. 


1 
P(2, 22) = (270) ee 


ee 


(12.107) 
This Eq. 12.107 shows that p(z;, 22) can be calculated if we know all the m,nth order 
moments of this probability function. 


On the other hand, by the definition of the cumulant 4,,, and the cumulant 
generating function K(i6;, i62). 


K(i61, 102) = log (61, 82) = at (16 :)(G62) + 2 (16;)(162) + 


Konan SRG PE TAMEATT 
= Sey 81) "(i82)". (12.108) 


Therefoie. 
2 2 kinn ° m5, nN 
(61,2) = exp{K (1, 102) = no So (162) 
mn min: 
kon 1 ; 
=1 {54s sa - - 8 rosy} Ai i {de ar warvo.r| 


tof oe oo ee 22109) 


When we insert this @(@;, 02) into Eq. 12.106, 


1 
P(21, 22) SCE | J elton. io) —1012. + 0r20}ae 


rai (12.110) 
J | exp <2 (i63)" (62) - oO:Z1+9222) 99 dO. 
Se & min! 


From Egs. 12.104 and 12.109, kp, and Um, are related as follows: 
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(order in €) 


koi = Lo1 eo e( (1): 

kon = Mor — M1 oe (EES 

ko3 = Mos — 3uoitlen + 2u4G; 22 G82) 

kio = 10 a (2). 

Ky = 411-1001 4 ey ()) (12.111) 
kz = M12 — Wigton — 2uo11 + 21045; pe(ecee) 

k0 = M20 — Mio ... (+e) 

koy = M21 — 21011 —Motl20 + 27 qulo1 »4 =) 

kao = 30 — 341020 + 23 (eRe) 


Following these general formulations by Vinje, M. Hineno®’ calculated the proba- 
bility distribution function of the wave height, treating the wave process as nonlinear, as 
was mentioned in Section 9.2.1, and will be summarized as follows. 


From these relations with u,,, the order of kp, in€’s was obtained, which is a 
smallness parameter that appeared in Eq. 12.100 as is listed in Eq. 12.111. Expanding 
Eg. 12.110 arounde =0 into a Taylor series and truncating at O(€) gives 


1 1 1 = 
p(21, 22) = =| | exp — k20(0)81- ko2(0)63 


x [1+ hotoo) += Kso(0)(0 1)? += kox(0)(i82)° 


= Kt 0G0 1083) | eH9:2:+6222) dO dO>, (12.112) 


: re) 
where k;j(0) = ben k;(0) = =e ke 


€=0° 


After manipulation, 
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1 a TS 
P(21, 22) = ——7==— ex ESE ip ae 
2at Jokes >| 2kag 2) 


A 


omy | actu ae 2 
Vk} 6 koavkon ~ \ Vkoo 


an| 
a 
S a] 
tom 
a 


ki0 Z1 
x |H5+—— H, | — ] + 


€ kip Zi 22 
+— H, | — | @, | —— ]]. (2.113) 
Here H,,, (x) is a Hermite polynomial, 
1 Up ; 
H,(x)=—= +i/20)"dt 

(x) iE Je (x+1V2t) 
H (x)= 1 
Hy (x)=x 
Hy(x)=x-1 
H3(x) =x -3x 
A n+) (&) = XH, (x) — nH) (x) (12.114) 


convention of (0) is omitted. 


Under the assumption that z,(r) is narrow banded, the number of zero—crossings of 
an output z(f) is equal to the number of maxima. 


Therefore, the expected number of z—up crossings per unit time is given as already 
shown in Section 12.5, Eq. 12.44, 


E[N,(z1)] = | lzo| p(z1, 22)d22. (12.115) 


N]e 
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Also, the expected number of mean level up—crossings is 
loo} 


1 
EIN.) = 5 | lzo| p(Z, 22)dz2. (12.116) 
The probability distribution function of a maxima then lies above Z; is 
E[(N4(z 
P(z1) = ay (12.117) 
E[N.(21)] 
Substituting Eq. 12.113 into Eq. 12.115 and carrying on the integration gives 
1 [kg Ze 
E[(N.(z)] = — foo exp _———— 
2m 20 2k20 
ko Zz £ ko Z1 
S| apa 18 hy || SS ea 
vk20 5 kao Vk20 Vk20 
kK 
ee Ay hae) 


€ 
+> oo 
2 korVkoo  \ Vka0 


As already shown in Eq. 12.111, the cumulants are calculated from the m,nth 
moment of response, and the m,nth moment can be expressed by using the frequency 
functions and the input spectrum as some of the examples that were shown in Section 


11.4. With the order of each term up to 0 €) taken into account, and with the spectrum 
function, which is a real function, the cumulants can be written as follows: 


359 


kio = €hio=€ | S(w) G2(w,—w)dw 


—c 


ko9 = hao = | sw IG1(@) do 


kon = hon = | ws(w) |Gy(w)"dw 


ky2 = €hy2 =€ | | (o102+20% 5(@1)s(W2) 


X [Gi(—@1)G1(— @2)G2(@1, @2) 


+ Gi(—@1)Gi(—@2)G7(@1,@2)] dw dw 


8 


k39 = €h39 = 3€ | S(W1)(sw2) 


8 


X [G1 @1)G1(— @2)G2(@1, w2) 
+ Gi(—@1)Gi(—@2)G2(@1,@2)] dw dw, 


where * indicates a complex conjugate, 
S(@) is the two-sided spectrum of input x(2), 


G(@) is the linear frequency response function 


G\@) = | gy(te"@ dr}, 


—c 
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(12.119) 


and G2(@), 2) is the quadratic response function 


29(T), Tr )€ Ord dT. 


——_ 8 


G7(@1,@2) = | 


8 


Through these manipulations, Hineno obtained his expression for the probability 
distribution function as 


2 h h 2 h D) 
P(z) = exp] -—— <0 | 30 | = 3 } +— 42). 12.120) 
2h20 hyo = 6h \ h20 2ho2h20 


The probability distribution density function p(z) and the expected 1/n highest value 
Z1/n are obtained from the relations 


p(z) = “ey @2s2 1) 
dz 
| z p(@)dz eo 
Z1/n = a =n | z p(z)dz. (12.122) 
{ P(z)dz eae 
Z1/n 


If we use Eq. 12.120 for P(z) and consider that in real calculations, the smallness 
parameter € is absorbed in the computation of G2(w ,w2) and does not appear explicitly, 


fe Zi hijo _ h30 hy 1 
PU) expla nn ae pee 
2h20 hoo =2h3g = 2h20ho2 = h20 
1 h h h h h 
— | 0 | AL 2.123) 


This is the final expression of the probability distribution density function of the maxima. 
Using this expression, M. Hineno calculated the probability characteristics of the maxima 
and minima of the waves that were treated as nonlinear with the quadratic response 
function, as shown in Eq. 9.10 in Section 9.2.1, 
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G2(@ 1,2) = os (03 a 3) 
G2(@1,-@2) = pols (0? - 3) 


with the linear response assumed as G;(m) = 1. 


Fig. 12.13. Probability density function of the maxima of waves. 
(From Hineno.®7) 

Figure 12.13 shows the probability distribution density function for maxima of the 
waves compared with the experimental data by a certain research worker for model 
waves with a wave spectrum that is almost of the Pierson—Moskowitz type. In Fig. 12.13, 
the + marks indicate the experimental data analyzed as nonlinear waves and the other 
marks (A, O) are experimental data analyzed on the assumption of linear waves, the solid 
line being the theoretical relations as linear for this model waves. Results computed by 
Eq. 12.123 are shown by a dotted line, and the agreement with + signs is quite good, 
especially in the larger amplitude range, which is important practically. 

Figures 12.14, 12.15, and 12.16 illustrate the calculated results for nonlinear waves, 
using a modified Pierson—Moskowitz wave spectrum with H/3 = 11.6 m, To = 16.1 


sec as the input. From these figures, we can find the extent of nonlinearity of the waves 
and also the effects of nonlinearity or the effect of the distortion of the wave forms on the 
difference of the probability distribution function for maxima and minima. 
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Hineno®® also calculated the relative motion of a semi—submersible in this kind of 
nonlinear wave, assuming a linear response to the excitation by the waves. 


Hy = 11.6m 
To = 16.1 sec 


— MAXIMA 
—-— MINIMA 
——- LINEAR 


Fig. 12.14. Cumulative probability distribution function of wave amplitude. 
(From Hineno.®’) 


Hig = 11.6m 

To = 16.1 sec 
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—— MINIMA 
—— LINEAR 
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Fig. 12.15. Probability density distribution function of wave amplitude. 
(From Hineno.®) 


12.7.2 Wide-—Banded Case 


J. F. Dalzell** (1984) extended this technique further. He did not assume narrow 
bandedness of the response and did not truncate the functional polynomials at n=2 but 
continued to n=3. He thus formulated the technique for calculating the probability distri- 
bution function of extremes of the nonlinear responses to Gaussian inputs. Here the 
characters of the nonlinear frequency response functions up to degree 3 are assumed to be 
known from the analysis as discussed in Section 11.5. The nonlinear response Y(t) to 
Gaussian input X(r) is expressed by 
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Hin = 11.6m 
To = 16.1 sec 


— MAXIMA 
——— MINIMA 
—— LINEAR 


1 © 5 10 15 20 


Fig. 12.16. Expected 1/n highest values of wave amplitude. 
(From Hineno.®’) 


Y(t) = [ een X(t—T) dt, 
+ | | ex 1,72) X(t—T1) X(t—T2) dt\dr2 
+ | | | exer.) x(t—T1) x(t—T2) x(t—73) dtydt2dt3. (12.124) 


(Limits of integrals — © — + o are omitted throughout this section.) 

As was assumed in Section 11.5, here the kernels or the nth degree impulse 
response functions g,(Ti,T2 . . . T,) are real, time invariant, completely symmetrical in 
the variables g,(t),T2 . . . Tr) = p(T273 . . . Tr, T1) = . . . for any rearrangement of the 


variables t;, and sufficiently smooth and integrable so that there exist n—fold Fourier 
transforms, 


&n(T1,T2 Sion maa | | coe | Gx(o1,07 0 0-0 On) 
a 


n 


n 
exp |S nao, ... dW, (12.125) 
j=l 
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G,(@1,@2 .. . Wn) = | et | entera, 306 Gy) 
[ee 


n 


exp LS enfin dr (2126) 
j=l 


The function G,(w},@2 .. . @,) is the nth degree frequency response function and 
is also symmetric in its arguments G,(@1,@2 .. . Mn) =Gp(W2,@), ...@,)=... for 
any rearrangement of w; because the impulse response functions are real. 

G,(—@1,-@2, Oy 0 —OW,) = G7(@1,@2, oe Wn) (12.127) 


here the * denotes the complex conjugate. 
In Dalzell’s paper,>° the spectrum S,(w) was defined a little differently from those 
used by this author in Parts I, II, and II. He used 27 times our s(w) , 
S,A(@) = 27s(w), (12.128) 


and also took the one-sided spectrum 


U;,[|@|] = 2s(@) = = 5.(w), forw@ > 0. (12.128’) 


The initially assumed functional polynomial process was reformulated as the 
response to a white noise excitation, and a new set of frequency response functions was 
defined which contains both characteristics of the original frequency response to the 
exCitation and that of the excitation spectrum. 


The two-sided spectrum of white noise is 
Sea ile (12.129) 


and the autocorrelation of white noise is a delta function 


Ry, AT) = = | exp[iwt]dw = 6(t). (12.130) 


We think of the spectrum filter L(w) that expresses the spectrum of input S,.(w) 
with the white noise as 
Six(w) = 0? Lo)? (12.131) 
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ISAC 


Ox 


Li) (12.131’) 


The linear, quadratic, and cubic frequency response functions connecting the white 
noise W(t) with the output Y(t) will be 


0,H\(w) = 0,L(w)G(w) 
07H2(w1,@2) = 02L(w1)L(w2)Gx(@1, @2) (12.132) 


03H3(@1, 0203) = 02L(@1)L(w2)L(W3)G3(@1, @2, W3). 


Then Y(t) can be related to the white noise input W(t) as 


Y(t) = Ox | hy(t)W(t—7))dry 


+02 | | hy(T1, T2)W(t— 71) W(t — T2)dt ar 


+o; | | | h3(T1,T2, 73) W(t — 71) W(t — T2)W(t —13)dt\dt2dt3, (12.133) 


where 
h(t) = = | H\(w) expliwtjdw 
1 
h2(T1,T2) = so | | H12(@ 1,2) exp[i(@ 17 + W2T2) dw dw (12.134) 
1 
h3(T1, 72,73) = Om: | | | #01,02,03 exp[i(@ 17] + W2T2 + W373) ]dw\dw2dws. 


The derivatives of the output are then 


Y(t) = 0, | hy(t)W(t —1))dr1 


+05 | | hy(t1, T2)W(t— 7) W(t — 12)dr dt 
+o; | | | h3(T,72,73)W(t—1;)W(t —T2)W(t—13)dt\dt2dt3, (12.135) 


where 
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5 It ; 
hy) = = | H\(@) exp(iwt)dw 


h(t1,72) = | | Aoto:.02)explivw in + w2r2)ido a (12.136) 


1 
(250)? 


h3(t1,72, 73) = | | | He. 2,3) expli(@ 171 + W2T2 + W3T3)]dw \dw2dw3. 


1 
(22)? 
The terms H,,(w) are defined as 
H\(@) = iaH\@) 
H>(@1,@2) = i(@ + @2)H2(@1, 2) (12.137) 


H3(@ 1, 2, @3) = i(@1 + W2 + W3)H3(@1, W2, 3) 


and 
Y(t) = ox | hy (ty) Wt —1))dr 
+ oe | | ho (T1, T2)W(t —T1) W(t —T2)dt1dt2 
+ oe | | | hg (71,72, 73) W(t —T1)W(t —T2) W(t —73)dt\dt2dT3, (12.138) 
where 
, 1 , : 
hy (11) = se | Hi (w) exp(iwt;)dw 
; 1 on 
hp (T,T2) = Ome | | H2(@ 1, @2) exp[i(w jt; + @2T2)|dw \dw> (12.139) 


h3(t,,72,73) = | | | H3(@ 1,2, @3) expli(w yt) + W2T2 + W373) ]dw dwrdws, 


(2x)? 


and H, are defined as 
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H\(@) =-o7H\@) 
H>(@1, 2) = —(@1 + @2)"H(W1, 02) (12.140) 


H3(@1,@2, @3) = —(@1 + @2 + ©3)"H3(@1, @2, 3). 


In addition, the characteristics of the products of white noise were fully utilized as 
follows: 


Wi (t—t)W(t—t) . . . Wit—ty) = 0 for N odd, 
W(t—1)W(t—72) = 0(t1 —T2) = O12, 
W(t=1)W(t—12)W(t— 73) W(t — 4) = 012034 + 613024 + 6 14623, 
W(t—1)W(t—12)W(t—73)W(t — T4) W(t — 75) W(t — 7.) 
= 612034056 + 012035046 + 012036045 (4 141) 
+ 613024056 + 013025046 + 013026045 
+ 614023056 + 614025036 + 014026035 


+ 615034026 + 015032046 + 015036024 


+ 016034052 + 016035042 + 016032045, 


where 6;; = 0(t; —T;). 
Eq. 12.141 is the special case of Eqs. 11.68, 11.69, and 11.70 in Section 11.5. 


If we do not assume narrow bandedness of the output spectrum, the expected 
number of maxima of the response Y greater than Y = € per unit time is approximated as 


«2 0 
Nz = | | IYI p(Y,0,¥) d¥ dy. (12.142) 
— -w 


Similarly, the expected number of minima of Y less than Y = € per unit time is 
approximated as 
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Eo 
Nz = | | IY] p(¥,0, ¥) dY dY. (12.143) 
-< 0 
Then, the expected number of maxima regardless of magnitude per unit time will be 
o 0 
na= | J IY] p(Y,0,¥) d¥ dy. (12.144) 
Similarly the expected number of minima per unit time will be 
Ne = [ 
-o 


Because maxima and minima are paired in the same record of response N a= NG, 
from Eqs. 12.142 and 12.144, the probability that a maximum will be less than € is ap- 
proximated 


Il p(¥,0,¥) d¥ dY. (12.145) 


ou 8 


N 
Prob[Maximum < &] = 1- = (12.146) 
Similarly, 
sea 
Prob[Minimum < &] = at : (12.147) 


Then, the probability densities of maxima and minima are obtained by differentiat- 
ing Eqs. 12.146 and 12.147 with respect to &, as 


pé)= 


INE 
No 


0 
= | IY p(E,0,¥) d¥ (12.148) 


369 


pé) =| IY] p(é, 0, ¥Y) dY. (12.149) 
0 


In the same way as we did for the joint probability density p(Y, Y) in Section 12.7.1, 
relations between the joint moment generating function (it, it, it) and the joint cumulant 


generating function K(it, iz, it’) were used to give 
P(it, it, it’) = | | | p(y, Y,Y)exp[itY + it¥ + it¥] dy aY dY 


=1+>, ea Gt) Ge)”. (12.150) 
& jim! 


Here {jem is the joint moment 


[TS VONE = | | | yy'y"py,Y,¥) dY dY d¥ (12.151) 


mice 5 BD 60d Kiem put eat ise <2" 
K(it, if, i?) = log (it, it, i?) = > Fe Gi) GFY". 42.152) 
PRINS aes 


Therefore 
P(it, it, if) = exp [K(it, it, if )] (12.153) 


where j, k, and m are positive integers whose sum is greater than zero. Therefore, the 
inverse transform of Eq. 12.150, from Eqs. 12.150 and 12.152, is 
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exp [- {i¥e + i¥t— iv} (it, it, it ) dt,dt,dt 


exp |—{i¥e+ tit—i¥t]] exp [KGr.itit)] dt.dt.a 


exp [-i¥r—i¥r—i¥s+ K(it, it, it)| de,de,ar 


| ex [100 ~Y) it+(ko10—Y) it + (koo — Vit 
1 9 35) : x 50 
5 {ikooot? + Ko20t~ + Kooat “+ 2k1 ott + 2kio it t+ 2ko yt t } 


+> ae — (ity) (i) GEY") dt, di, di (12.154) 
< jtktm! 


Here j, k, and m are now positive integers whose sum is greater than 2. 


The relations between joint moment and joint cumulants are 


k100 = 4100 
ko10 = Ho10 
koo1 = Ho01 Ones) 
ko09 = variance of Y ve 
ko29 = variance of Y 
Koo2 = variance of Y. 
Nondimensionalizing the variables as follows 
1/2 
z=(Y- k1o0) kobe 
. 3 1/2 
Ba (Ve ain (12.156) 


Z = (¥—kookohp 


and 
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1/2 


= t ky09 

s=7KR (12.157) 
e=7 KY 

and 

Kjkm 

Mn a, 12.158 

eS ea ee, ER) Core 

pi2= see | | exp |-izs— 25-23 
(27) 
-5\° + Se4 57+ 21.1108 § + 21015 § + 2o11S i| 
+ > Ajem(isy (is) Gs)" ds ds ds. (12.159) 


jkm 


Expanding the characteristics of the moments as shown in Eq. 12.159 into joint 
moments and modifying them into joint cumulants, Dalzell derived the expression for 
joint cumulants until the fourth degree from the modified frequency response character— 
istics H\(w), H2(W1@2), H3(@),W2W3), and functions of w by the same type of style as 


Eq. 12.119 for the case of p(z, z). Since these manipulations require many transactions 
involving the higher order terms in the expansions, he checked the order of magnitude of 
the functions, suggested the order to truncate the approximations and, by laborious 
manipulation utilizing Hermite polynomial, he obtained his approximation. 

From these expressions, arbitrarily denoting the standardized maxima or minima of 
response by v he finally derived the expression for p(v,0,z) using the cumulants up to 
the fourth order and then the probability distribution function for maxima and minima by 


0 
pew) == | [21 p(v, 0,2) az 
0 
== | Z p(v,0,z) dz (12.160) 
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p(vy= =| zl p(v,0,z) dz 
0 


in} 


1 
S|] Fp@ ODE (12.161) 
N | p(v, 0,z) 
0 


Finally he derived the expressions for his first and second approximations pj(v), 
Pi(v) and p3(v), p2(v) as follows. First, by neglecting the term higher (than 3) order 
joint cumulants, the first approximation is, 


ih Dee v Sim v v(1—€? 1/2 
Pi) =e exp] Fa +(1-€7)'/? y exp es anGer ae 
(12.162) 
2 2\1/2 
Ty ee Vis 21/2 v WL=Ge)) 
PAW) = in exp oe —(1-¢7)!/2 y exp} -— en erae 
(12.163) 


where #(a) is the Gaussian cumulative distribution function and € is the spectrum band 


width parameter. 
The € was introduced in Eq. 12.162 and Eq. 12.163 from the relations as follows that 


comes from the characters of joint cumulants, 


Ko20 21/2 
Ane 3 —— = Sil 12.164 
101 & 2 discs) ( ) 
1 =A%5) = €? 6 (12.165) 


It is interesting to find that this first approximation, Eq. 12.162, is just the same as 
the one that Cartwright and Longuet—Higgins!> derived as shown in Eq. 12.49, and Fig. 


12.1 in Section 12.5. 


Se 


Dalzell’s final expressions, including the effect of higher order cumulants (until 3) 
that take account of the higher order nonlinear response until the third order, are 


p3(v) = S(v,€) € 


2/2 
+ R(v,€) (1-e7)!/2 ex(-5] Q ae . (12.166) 


p2(v) = S(v,€) € Thae 


2) 2/2 
SI2O8) (ae we ox(—5] @ os 2.167) 


Here 


Asp + es = 5e") i Aoos(1 —€2 )!/2 


e+ «4 


SQ,Ee)=14+v 
Aio2 A201 


Sogn le el —€7) 4/2 _ Jiro 


+ v°A300, (12.168) 
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RO,€)=v+] Ayot+ + 34300 


A201 
él Est €2 yi/2 
2 A201 
cP 7 gaan 64300 


+ v2.30, (12.169) 


where e is the so—called band width parameter, as Eq. 12.165. 
As referred to in Section 11.5, item 6, Dalzell*° generated the simulation data of the 


cubic nonlinear response to check the validity of his tri-spectrum analysis!” for the 
Gaussian excitation of the waves, as expressed by the Moskowitz—Pierson type spectrum, 


- 1.25(33)' 


U,x(w) = 50209 exp———_. —— (12.170) 
@W 


where U,,() is a one-sided spectrum, o2 is the spectrum area and variance, and wo is the 
modal frequency. 

Using the same spectrum parameters for nonlinear response as he used before!’ and 
varying the excitation level by changing 0, tog, = 0.125, 0.25, 0.50, 0.75, and 1.0, he 
obtained results for simulated data averaging 10 samples for each case. 


He calculated the probability distribution of maxima and minima and compared his 
results with Eqs. 12.162, 12.163, 12.166, and 12.167. 

Figure 12.17 shows the comparison of probability distribution densities of response 
maxima and minima estimated for the simulations with those of the first and second 
approximations from Eqs. 12.162, 12.163, 12.166, and 12.167. Here z represents the 
normalized maxima and minima and was equal to v—Ajo0, A100 being the normalized 
mean and v the standardized maxima and minima as appeared in Eqs. 12.162 through 
12.167. 

Figures 12.18 and 12.19 compare cumulative distributions of response maxima and 
minima estimated from the simulations with those of the first and second approximations. 
In these two figures, the standard deviation of input 6, (“SIGMA’”) is the same, and the 
band width parameter, e, (“EPSILON”) was different. Here the first approximation is ex- 
pressed as a straight line (the scale of the cumulative probability is so chosen) and is 
marked as the fitted distribution. 


These figures indicate that the final expression for the second approximation gives 
excellent results that check the simulated results very well. 
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Fig. 12.17. Comparison of densities of response maxima and minima 
estimated from the simulations, with those of the 
first and second approximations. 


(From Daizell.5®) 
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EPSILON= .482 , SIGMA= 1.00 


Fig. 12.18. Comparison of cumulative distribution of response maxima and 
minima estimated from the simulation, with those of the first 
(fitted distribution) and second approximation — sample 1. 


(From Dalzell.5®) 
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Fig. 12.19. Comparison of cumulative distribution of response maxima and 
minima estimated from the simulation, with those of the first 
(fitted distribution) and second.approximation — sample 2. 
(From Dalzell.5®) 
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CHAPTER 13 


EXTENSION OF MODEL FITTING TECHNIQUES TO 
NONLINEAR PROCESS 


13.1 INTRODUCTION 
Suppose we have a discrete random process X; and a purely random process €,, and 


X; is expressed by 
X, = Qa(Xi-1, X12," + - Xin) + €2- (13.1) 


When Q, is a linear function, 
X,= a, Xp) + a2 Xp2+- > ++ Qn Xen t€:, (13.2) 
X, is called as an autoregressive process (AR) of nth order, as discussed in Part II, 
Chapter 5. 
If X, is expressed as 
Xr = Om(Er-1,€1-2€1-3,€-m) + Er, (13.3) 


and Q,, is a linear function, 
X, = by€,-1 + bz €-2+- + -+ Dn Epmt Er, (13.4) 
then X; is a moving average model (MA) of mth order, also discussed in detail in Part I, 
Chapter 5. 
When these Q, or Qj functions are not linear functions but, for example, the poly- 
nomials of X;_, Or €;» , then X; is no longer a linear process and is called an expanded 
AR model or an expanded MA model. 


A Voltera type process, as we saw in Section 11.1 in Eq. 11.1, is this expanded 
moving average process. More generally, 


X,= QaXi1 Xr-2, Xin) + Om(Er-1,€+-2€r-m) + € (13.5) 


is an expanded ARMA model when Q,, Q,, are polynomials of variables. 
For expanded AR, MA, and ARMA models, there is no general way to solve the 
process. When the process is expressed as, 
X,+4, Xp1+ 42 Xpoat+- > > +ay Xx 


m v 
=€,+ dy €3+: °° +b; €pj+ Sy Gi; Ei Xj, (13.6) 
i=1j=l 
Eq. 13.6 is called a bilinear model. The scope of nonlinear models that have been analyti- 
cally developed is rather limited. A few efforts have been made along the following 
models: i) a simple bilinear model, ii) a threshold autoregressive model, and iii) an 
exponential autoregressive model. 
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13.2 BILINEAR MODEL 
As the simplest example of a bilinear model, we take the first order model 
X141 = AX, + DE p41 + CE Xy (13.7) 


Priestley,~> in studying the bilinear model, assumed that Eq. 13.7 can be expanded into 
the form of Voltera functions, Eq. 11.8 as, 


X= » Suu + y ier Ue Uae 904 (13.8) 
u=o u=ov=o0 
Here, the general transfer functions are 
Ty(@1) = »> BO 
u=o 


oo fo) 


IT3(@1,@2) = S. Sy Luv QED | (13.9) 


u=ov=0 


Assuming Eq. 13.7 is expanded into the form of Eq. 13.8, he derived I"j(), setting 
€,= ein Eq. 13.7, as 


10 
T1(@|1) = ———_. (13.10) 
1(@1) (Ea) 
Similarly setting €, = e'?' + e@2", 
G 
T3(@1,@2) = eterno al {T(@1) +T(@2)} 
Cerri -a} 
1 iw ww 
im =cb e1 e2 
{e wren) —a} e_q e?_a@ 
Particularly along the diagonal w, = w2=a, 
che 
I,@,0) =—>———_. - (13.12) 
2 ) (e* — aye” —a) 
More generally, Priestley showed that 
EMD G2 
TB OS) (13.13) 


(Chie 2) Cea) cn Con) 


From these relations, he found that '}(w1,w2) included all the a, b, and c of 
Eq. 13.7. Thus he says, the bilinear process expressed by Eq. 13.7 is invertible to the 
functional polynomial model of Eq. 13.8. 


This bilinear model can also be approximated he says, by the generalized autore- 
gressive model. The first order bilinear model can be expressed by setting b = 1 in 
Eq. 13.7 (without losing generality), 
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Xin] = AX, + CX, €;+€ p41. (13.14) 
We can modify this equation to 
{1 +X, Ble xX aXe (13.15) 
Here B is the backward shift operator. For small c, Eq. 13.15 can be inverted to 
emi ={1—c X, B}Xi41—aXi] 
SG SO SOG lO GF Xs (13.16) 


This is a generalized autoregressive model. If the product term X; X,_1, is neglected, initial 
estimates of a and c can be obtained by a least squares approach. If the product term is 
not neglected, a nonlinear least squares approach can be used to get a andc. After initial 
estimates dp and Cp are obtained, €, can be estimated by 


Enya = X41 — AX,;— CX, €; for t= ee -- + N. (13.17) 
Recursively starting from€, = 0, and using {X,.4] fort=1, °° ‘N, new estimates for a 
and c can be found to minimize 
N 2 
» {Kee —ax,-— X,e . (13.18) 


t=1 
Until the estimates converge, this procedure is repeated. Subba Rao®? studied this 
bilinear model and gave several examples. 
13.3 THRESHOLD AUTOREGRESSIVE MODEL 
The threshold autoregressive model introduced by Tong™ is generally expressed as 
GeO ante o oO San (13.19) 


() 
1 


where a”. - -a{ are constants in Region R® for 


OG SON. Gaal. Grea iige o 3/2 (13.20) 


Region R® is in the dimensional Euclidean space R’. For example, the first order 
threshold autoregressive model TAR(1) is 
- f DO GN Aan < a 
bce 9 


ae (13.21) 
aX, +e, #X,.2d 


where the coefficient a differs by the size of X,_). Priestley’ indicates that, if we consider 
a bilinear system in which the physical input U;, output X,, and noise €; are related by 


X41 = aX,+cU, Xp+ Epa 9 (13.22) 
and if U, is determined by a feedback mechanism of the form 
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U,={ "EM ee (13.23) 
-a ifX,2d 


then the model X; can be expressed by a threshold model 


(1) 
Ife X,t+ Er, for X; <d 
Keele 170) XGCHE ee LORXGue= a ae 


where 


GD) 
{ 20 =a+ac (13.25) 
a’=a-Q Cc. 


This model gives a nonlinear process as shown by Eq. 13.22. It can be called a piece— 
wise linear approximation. As will be shown later in Section 13.5, Ozaki?! made it clear 
that this piece—wise linearization is a special case of his nonlinear threshold autoregres- 
sive model. 


13.4 EXPONENTIAL AUTOREGRESSIVE MODEL 
The nonlinear random vibration system of one degree of freedom is expressed as 


X(t) + 21{X(0)} - X@) + 2o{XO} XO = n(1) (13.26) 
where n(f) is a random noise excitation. 
When 
X@ = YO, (13.27) 
Eg. 13.26 becomes 
¥(t) =— go{X()} X()— 21 [X(D)} YOO). (13.28) 


From these two equations and the state space expression as explained in Section 12.3 


X(t) 0 1 X(t) 0 


Yi) |=|-g.xo} -six@}| | vo|* | no G32”) 
Or, in vector notation, 
V(t) = f{xeo} V(t) +n(0), (13.30) 
where 
V(t) = ah = a (13.31) 


So, finding the appropriate expression for g)(-), 22(-) in Eq. 13.28 is the same as finding 
the appropriate function f {x@} for Eq. 13.30. When these two functions g)(-), g2(-) are in 
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AMPLITUDE 


the special forms as follows, the system is called a Duffing type or a Van del Pol type 
oscillator, respectively, as described in Chapter 10. 


Duffing type: 

gi\X(p} =c 

g2{X(t)| = a + BX? (13.32) 
Van del Pol type: 

gilX(0)} = —c{1-x(er| 

g2(X()} = a. (13.33) 


For the Duffing type oscillator, under harmonic excitement F coswt, 


X(t) +cX+aXx + BX? = Fcosat. (13.34) 


We know that, with harmonic excitement, the response shows damping phenomena, 
as indicated in Fig. 13.1, and in this case the natural frequency is dependent on amplitude. 
So we can call this oscillation an amplitude—dependent period shifting oscillation. 


SOFT SPRING TYPE 


HARD SPRING TYPE B<0 
| B>0 ~~ 
Ww 
Q 
E y) 
a) 
< 
ayy — Wo —>- o 


Fig. 13.1. Duffing type oscillator. 
For the Van del Pol oscillator, 


¥-cl1-X*}X +x =0, forc >0,a>0 (13.35) 


the system possesses a limit cycle, because when the amplitude X is small, the damping 
becomes negative and X starts to diverge, and when X(t) becomes large, the damping be- 
comes positive and the amplitude starts to decay. This system remains in oscillation 
without any excitation. 


When we have random noise excitation n(r) 
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X-—c(1-X2)X + aX =n(0), (13.36) 
the system will produce a perturbed limit cycle. 


Ozaki? !-93.96.97 and Haggan and Ozaki™?> proposed a new type of nonlinear model 
called an exponential autoregressive model through the following considerations. In Eq. 
13.26, when g, and g> are constants, the system becomes a purely linear oscillator and the 
equation of motion is expressed as 


X+cX+aX=n(t). (13.37) 


This expression can be inverted into an ARMA(2.1) model, as was explained in some 
detail in Section 6.3.2 of Part II, as 


X,= PiX-1 + P2X1-2 + O1€-1 + €2- (13.38) 


Here n(t) is a continuous Gaussian white noise and €,; is a discrete Gaussian white noise. 
The oscillation expressed by Eq. 13.37 is governed by its characteristic equation, as dis- 
cussed in Section 6.3.2, 


w+cu+a=0 (13.39) 
and by its roots (or eigenvalues) 
c : 
pone iva—(c?/4). (13.40) 


We know this model X, diverges when c < 0 and converges when c > 0. On the other 
hand, the model expressed by Eq. 13.38 diverges when the roots of its characteristic 
equation 


2-12-92 =0 (13.41) 


are outside the unit circle and converges to a stable process when two roots of Eq. 13.41 
are inside the unit circle, as described in some detail in Sections 5.2.4 and 5.2.3 and in 
Fig. 5.16. 


From Eq. 13.41, when @? < 4@, i.e., when the roots are unequal and complex 
(conjugate to each other) zone [ II ] in Fig. 5.16, 
1 
z= fhe 5 iy¥—402-?, (13.42) 
since IzI* = @2, when—@2 > 1 org <—1, this model diverges and when 1 >—¢2 > 0 or 
0>2 >-1, this model becomes a stable process. 


From the discussion of Green’s functions of ARMA(2.1) in Section 5.2.4 or of the 
autocorrelation of AR(2) in Section 5.2.3, we know that damping is determined by 


/—@2, i.e., by p2, and the frequency is dependent on@). 


In the analysis of a ship’s nonlinear rolling that comes from the nonlinear restoring 
moment (the Duffing type), Ozaki and Oda” first fitted a nonlinear model of the type 


X,= 1 Xi1+@2 X-2+H Xeyte. (13.43) 
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Equation 13.43 can be transformed into 
X,= (6, +2 X*_) Xi1t+o2 Xi2+&. (13.43’) 


This expression shows that the coefficient of X,;, the first autoregressive coefficient, 
changed from @, in Eq. 13.38 to(@,; +2 X2,) in Eq. 13.43’. This means the frequency, 


determined by the coefficient of X;_), became amplitude dependent. Although this looks 
fine, Ozaki and Oda found that the time series X,, defined by the nonlinear autoregressive 


model Eq. 13.4’ was not stable but could be diverged. Therefore, instead of Eq. 13.43, 
they proposed 
_V¥2 
X,= (git me) X-1+ 2 X-2+€:. (13.44) 
This is the exponential type of autoregressive model for a Duffing type oscillator. By ex— 
pressing it this way, we can make both roots A(0), A(0) of the instantaneous characteristic 
equation, when X,_; = 0, 


J? — ($1 +) -$2 = 0 (13.45) 
and also the roots A(), A(©) of the instantaneous characteristic equation when 
Xi] —> © 
17-6 -62=0 (13.46) 


lie inside the unit circle as shown in Fig. 13.2. Accordingly, with Eq. 13.44 as the model, 
we can express the amplitude—dependent and stable oscillation under white noise oscilla- 
tion excitation. 


Fig. 13.2. A(0), A(%) for stable Duffing type model. 
When 2, > 0, the model corresponds to a hard spring system and when 7 < 0 the 


model corresponds to a soft spring type oscillator. Haggan and Ozaki (1979)™ and 
Ozaki?’-* showed an example of this type as 
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X, = (1.5+0.28 e*) X,,-0.96 X,5+€, (13.47) 
where o7 = 0.025, namely €, is N[0, 0.025]. 


The eigenvalue A stays Ul? = 0.96 for 25° < Arg. (A) < 40°, and actually 
moves between Ap = 0.89 + 0.41i and A. = 0.75 + 0.63i when |X,_;| changes between 
0 and © as schematically shown in Fig. 13.2. The generated time history is shown over 
t = 1-100 in Fig. 13.3. To check the characteristics of this model expressed by Eq. 13.47, 
the model 


X; = (1.5 oP 0.28e7 -1) Xi) Im 0.96 Xi-2 + Qa: (13.48) 


was simulated,!© where a, is no longer random but is the form a; = sin{27 fit): th. where 
the frequency f(t) changes with time. 


4.00 


uit Ayal i Wi Uys ld i Wylie Hh 


00 100.00 200.00 300.00 400.00 500.00 600.00 700.00 800.00 900.00 1000.00 


o—4.00 0.00 


Fig. 13.3. Generated exponential AR model 
= (1.54 0.286%) X,, -0.96X-0+€;, €1: N[O, 0.025]. 
(From Ozaki.9) 

Figures 13.4 and 13.5 show these simulations. In Fig. 13.4, the frequency fiz) 
increases with time from 0.005 to 0.1, and in Fig. 13.5 the frequency f(t) decreases with 
time from 0.1 to 0.005. In Fig. 13.4, the amplitude suddenly becomes small at around 
f= 0.062; in Fig. 13.5, the amplitude changes abruptly at around f= 0.052 and actually 
demonstrates the jump phenomenon shown in Fig. 13.1. 


In the same way, by making the damping amplitude dependent, Ozaki,°** proposed 
the model for expressing the Van der Pol type oscillator as 


or, more generally, 


X= (91+ & *1) Xi-1+ (2+ me*1) X2+€1. (13.50) 
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INPUT a, 
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: H | | ! A uy 
ee MM ied 


Be Rae 


OUTPUT X, 


8.00 


SS 
SS 


00.08 0.00 


0.00 100.00 200.00 300.00 400.00 500.00 600.00 700.00 800.00 900.00 1000.00 


Fig. 13.4. X,=(1.5+ 0.286%) X1—0.96X,0+4:, 
Input a, = sin{sinz Kt): 4; f(t) is increasing by time. 
(From Ozaki.98) 


In Eq. 13.50, the coefficients 1,2, 01,72 are to be chosen to make the damping 


coefficient negative, when the amplitude X;_, is small and the oscillation starts to diverge, 
and to make the damping coefficient positive, when the amplitude X;,_, is large and the 
oscillation starts to decay. Mathematically 


A? -~(b1 + 1)A—- (G2 + M2) = 0, (13.51) 


has roots outside the unit circle, and 


1° -oA-$2=0 (13.52) 


has roots inside the unit circle, as shown schematically in Fig. 13.6 
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OUTPUT X 
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—e t 


Fig. 13.5. X, = (1.5+0.286%) X,,-0.96X0+a,, 
Input a, = sin{2x f(t) - t: Kf) is decreasing by time. 
(From Ozaki.98). 
As an example, Ozaki showed that, for a model 
X, = (1.95 + 0.23 e-%1) X,1- (0.96 + 0.24 eX) Xin+€;, (13.53) 


the characteristic roots are Ag = 1.09 + 0.109i and A~ =0.975 + 0.0968i, and LX!? 
actually moves between lol? = 1.20 to U.|* = 0.96, which shows Van del Pol type 
oscillation, as shown schematically in Fig. 13.6. Figure 13.7 shows that the model 
without €,, which starts with different initial values, approaches to the same limit period. 
Thus this shows that the model expressed by Eq. 13.53 has a stable limit period. 


Fig. 13.6. 1(0),A(«) for Van del Pol type model. 
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Fig. 13.7. X;= (1.95 + 0.23671) X,, — (0.96 + 0.246)X, 2, with different 
initial values (without white noise). 
(From Ozaki.98) 


More generally, Ozaki® proposed the higher type Duffing and Van del Pol nonlinear 
model as 
X,= 91(Xr-1) Xr1 + B2(X-1) Xr2 + €r, (13.54) 
where 
(X11) = git (xg? Xt Xe ni))x",) exh 


22(X1-1) = 2+ (af? + TOaxXen + Wx? | aro © OCs 7 Xs) exe, (13.56) 


The term g(-) is expressed by a constant plus a Hermite type polynomial. When 
r,; = 0,72 = 0 in these equations, an equation of the type of Eq. 13.50 is obtained. 


To determine the order, Akaike’s criterion (AIC) was shown to be applicable for 
these nonlinear models, too. 


(13.55) 


13.5 NONLINEAR THRESHOLD AUTOREGRESSIVE MODEL 


Ozaki®? extended and generalized his model further and proposed his nonlinear 
threshold autoregressive model and a unified explanation of the nonlinear models. 


For example, the exponential autoregressive model that he used as an example of a 
Van del Pol type nonlinear process, Eq. 13.53, 
X, = (1.95 + 0.236%) X,1— (0.96 + 0.24e%") Xp +e; 
was shown to be approximated by Tong’s linear threshold autoregressive model, as 


x = 1.95 Xi-1 4 0.96 X12 + €; if X,_1! 20.5 (13.57) 
2.18 X;1—1.20 X2+¢€, if X11 < 0.5. 
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The characteristic roots of the model jump from Ao| ol? = 1.2 | to 


A al ols 0.96 or vice versa as in Fig. 13.6 and also shown by a step function in 

Fig. 13.8. While in the model as is Eq. 13.53, the path of two rootsAp andA and 4 at 
arbitrary X,_; is specified by a continuous Hermite—type polynomials in the equation, as is 
shown by a continuous curve in Fig. 13.8. 


Al? 


05 1.0 Ka 


Fig. 13.8. Path of characteristic roots of threshold model and 
exponential AR model. 
(From Ozaki.99) 


Ozaki argued that the threshold linear AR model, however, could not be expected to 
give a good enough approximation to nonlinear vibration, and appropriate nonlinear 
threshold AR model must be formulated. His point is, we have to consider the stepwise 
dynamics of restoring and damping force, but at the same time, the orbitally stableness 
and independence of the limit cycle of the Van del Pol equation must be maintained. 


Instead of the linear step function approximation, nonlinear approximation 
considered was as follows. 


r= 


{ 1.95 X,_1 — 0.96 Xp2+ Er, for IX ,-1! => 1.0 
(2.18 =0.23 X2,) Xe1—(1.2—0.24 X2) X,o4€:, 
tae (Osh 2 10) (13.58) 


Equation 13.58 can give the characteristic roots that move much more smoothly than 
those of Eq. 13.57. Thus, the general form of the nonlinear threshold autoregressive mod- 
el for nonlinear vibrations was proposed as, 


ae (ae ° “+@p Xipt€:, for \X,_4| =>T (13.59) 


where 


fA) = GP + aPX4-- +0 XM. 


i 


Schematically, the behavior of the characteristic roots for (1) linear threshold model, 
(2) nonlinear threshold model, and (3) exponential AR model are expressed as shown in 
Fig. 13.9. 
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(1) LINEAR THRESHOLD AR MODEL (2) NONLINEAR THRESHOLD AR MODEL (8) EXPONENTIAL AR MODEL 


Fig. 13.9. Schematical expression of behaviors of characteristic roots 
by Ozaki.99 


Ozaki! investigated the stability and existence of limit cycles and their relation to the 
form of the characteristic equation. For example, he showed that a nonlinear threshold 
model, 


oe 0.8 Xe stIGin for IX,_4| =>1.0 

(0.8 + 1.3 X2,-1.3Xt)) Xei+€:, for X,4! < 1.0 (13.60) 
has characteristic roots that behave as shown in Fig. 13.10. This model also has three 
stable singular points and gives a process that fluctuates around one of these points and 


jumps from one stable singular point to another, depending on the white input, as shown 
in Fig. 13.11. 


With these discussions and examples, Ozaki showed the greater generality of his 
exponential models. 


X; 


Fig. 13.10. Behavior of characteristic roots of a 
nonlinear threshold AR model. 
(From Ozaki.99) 
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Fig. 13.11. Time history of a nonlinear threshold model that has 
characteristic roots that behave as Fig. 13.10. 
(From Ozaki.29) 
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CHAPTER 14 


CONCLUSIONS FOR PART II 


The so-called “‘spectrum correlation method” discussed in Parts I and II has been 
shown to be a powerful way in the analysis of stochastic processes, but only when the 
processes are linear. Many steps in the manipulations in Parts I and Il are based on the 
assumption of linearity of the process. 


In the analysis of seakeeping data, as is usual in the general engineering field, many 
phenomena can be approximated as linear, and so spectrum or correlation analysis has 
played a significant role in advancing the technique of handling those data. Now, 
however, the nonlinearity of seakeeping data for ocean vehicles and structures has 
gradually been introduced, as mentioned in the Introduction. 


Here in Part III of this lecture, methods for treating the nonlinearity in the stochastic 
process analysis were summarized and reviewed. The conclusions obtained here are as 
follows: 


1. Several works on the effect of the nonlinearity of ocean waves on its spectrum 
were reviewed. It is now clear that, if necessary, we can get the effect of nonlinearity on 
its spectrum as well as on the probability distribution of the maxima and the minima of its 
amplitudes. It is also clear that the nonlinearity of ocean waves is usually quite small. 


2. Even when the nonlinearity of the waves is small, the response of ocean vehicles 
and structures might be nonlinear, because of the very low frequency characteristics of 
their responses that respond rather severely to the higher order nonlinear excitation by 
waves or because of the nonlinearity in their response characteristics. For these cases, 
when the nonlinearity is weak and the response characteristics are expressed by weakly 
nonlinear equations of motions, the equivalent linearization and perturbation methods can 
be applied if the excitation is approximately linear. The perturbation method was applied 
by this author rather early to ship’s oscillation, and now both methods are well formulated ~ 
as shown in Chapter 10. These methods are applicable to obtaining the first approxima- 
tion of weakly nonlinear damping and restoring oscillations under random excitation. To 
proceed to the second or third order approximation is, however, not necessarily easy. 


3. The Voltera or functional expansion is one appropriate way to express the weakly 
nonlinear response to random excitation, and the polyspectral analysis is a reasonable 
way to get the higher order nonlinear frequency characteristics. The procedure for this 
analysis was summarized in Chapter 11. 


4. Recently, J. F. Dalzell has engaged in significant efforts on polyspectral analysis. 
He tried not only bispectrum but even trispectrum analysis in the study of seakeeping in 
irregular waves, with excellent results, although an enormous amount of careful computa- 
tion was necessary which at this stage might not be feasible for practical purposes. 


5. Probability distribution of the maxima and the minima of the oscillatory motions 
can also be obtained from the nonlinear frequency responses. The general procedure for 
obtaining the probability distribution of extremes by calculating the cumulants related to 
the nonlinear frequency response characters and a few other examples are reviewed in 
Chapter 12. 
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6. If the process can be inverted into a vector Markov process by state-space trans- 
formations, sometimes the solution of Fokker—Planck equations can be used to estimate 
the joint probability distribution function, from which many probability characteristics of 
the original process can be derived, even for nonlinear oscillations. 


7. Recently, a few applications of this approach have appeared in seakeeping 
studies. However, the Fokker—Planck equations have been solved only for some limited 
cases, and are not so familiar to engineers. More studies are necessary for naval architects 
to become familiar to the application of this method. 


8. Some efforts have been made to expand the model fitting method to the nonlin- 
ear process. The threshold autoregressive model, the exponential autoregressive model, 
and the nonlinear threshold model are examples of expanded models. Some of these mod- 
els look promising and some are under active development, but we will have to wait until 
they are more fully formulated to accumulate experience in application to practical 
problems. 


As was mentioned in the Foreword, the contents of this report were summarized at 
the time of this author’s oral presentation at DTRC in July 1985, reflecting the state of the 
art up to 1984, though the written version was completed in August 1987. After these 
dates, the state of the art has made considerable progress, especially in the field treated 
here in Part III, and this author finds the ‘review,’ and this conclusion, to be insufficient 
because of the recent works of several researchers. In order to update the report, this 
author added a Supplement of References 101 through 124, listing publications that have 
appeared since 1984, together with some other publications that were not referred to in 
the original manuscript. 
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CHAPTER 15 


CONCLUSIONS FOR PART I THROUGH PART Ii 


Part I of this lecture note summarized the conventional procedures for analyzing the 
irregular time histories of observed data, like irregular ocean waves, and the responses of 
marine vehicles and structures on the ocean, that can be treated as weakly stationary er- 
godic stochastic processes. In these analyses, the so-called Wiener’s general harmonic 
analysis technique plays a large role, and the correlation and spectrum functions of the 
processes were important and included much information on the characteristics of the 
processes. When the responses of some dynamic systems to random excitation are to be 
treated, the cross relations of the input (excitation) and the output (response), i.e., the 
cross correlations and cross spectra, are very important. 


The theories of the analysis are complete and rather beautiful. However, in sample 
computations from practical data, that is, from discrete data sampled at some time inter- 
val of finite length, many statistical considerations are necessary in estimating the 
correlation and spectrum functions to get statistically reliable results. After the general 
procedures were summarized, the discussion concentrated on that point. In these proce- 
dures, this author made several suggestions for improving the reliability (actually the 
coherencies) and these results were summarized. This author stressed that we should pay 
more attention to the time—domain characteristics of the functions. For example, the cor- 
relogram, that is the diagram of the correlation function and is a function in the time 
domain, deserves as much attention as is now paid to the spectrum. 


Part II is concerned with the parametric analysis of a stationary process, which is 
really an alternative or more modern method for analyzing the sample random process. 
This author believes that the characteristics of the function in the time domain play bigger 
roles in this method than in the nonparametric conventional method, discussed in Part I. 


The parametric method fits a statistical model to an observed process and estimates 
the parameters from the observed data. In Part II, the autoregressive (AR) models, mov- 
ing average (MA) models, and mixed autoregressive moving average (ARMA) models 
were introduced. Since these models and the parametric approach are not familiar to most 
engineers, especially in the field of naval architecture, the author explained them in con- 
siderable detail. (He is afraid it was in too much detail.) 


Usually a moderate or rather low order finite AR(n), MA(m), or ARMA(n,m) model 
can be fitted to represent adequately most of the observed processes. The optimum order 
of these fitted models, n or m, can be estimated by a method called Akaike’s information 
criteria (AIC). This method is based on information theory and enables one to choose the 
order n or m that maximizes the statistical entropy of the estimate. It is therefore called 
the ‘Maximum Entropy Method,” although it is different from a similar method already 
published under the same name. The AIC method can give the optimum order to be 
adopted. In Part II, in explaining the characteristics of these models, this author generated 
several fundamental models, the AR(0) (pure random), AR(1), AR(2), ARMA(2.1), 
MA(2), MA(1), and ARMA(2.2) models, by simulation and analyzed them by the para- 
metric method. The orders were estimated by AIC and, for almost all processes, we 
succeeded to fit models that coincided with the models used to generate the simulated 
process. In these demonstrations, the simulated processes were also analyzed by the 
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nonparametric conventional method (sometimes called the correlation method, or the 
Blackman-Tukey, BT method), and the spectra were compared with the ones obtained by 
the parametric method. From these comparisons, we can get an idea of the characteristics 
of the parametric method. Although they could have very steep peaks, the spectra from 
the parametric method are smooth, and are free from the erroneous smoothing (blurning) 
effect of spectral windows in the nonparametric method. They are differently effected by 
the statistical fluctuations of sample estimation that come from the finiteness of discrete 
data. 


Examples of results obtained by applying this parametric method to the analysis of 
seakeeping data for marine vehicles and structures were also given to demonstrate the 
usefulness of this approach in this field. 


This author concludes that although this parametric approach is not the ultimate 
method of course, it is very promising, gives reliable results, and offers a supplement to 
the nonparametric method explained in Part I. The nonparametric method is, in a sense, a 
method for estimating infinite or very large number of parameters from finite data, or for 
estimating their spectrum or correlation functions. The parametric method on the other 
hand is a method for estimating a finite number of parameters for the same purpose. 


The characteristics of a dynamic system, which is usually assumed to be determinis- 
tic, must be approximated by a finite number of parameters because the system is usually 
governed by equations of motions with a certain finite number of parameters. The types 
and also the order of parametric models are closely related with the order of the equations 
of motions, so the parametric method can be presumed to be more reasonable in the anal- 
ysis of response characteristics. 


In Parts I and I, the processes were assumed to be linear. In Part III, the method for 
treating a nonlinear process was summarized and several methods, such as the lineariza- 
tion method, the perturbation method, and the Voltera expansion method (polyspectra 
method), were reviewed and the relations of these methods to each other were described. 
Then, in relation to the probability distribution of the extreme amplitudes of nonlinear 
processes, the application of the Fokker—Planck equation and some examples appearing 
recently in naval architecture were introduced. The probability distribution of extremes of 
nonlinear response process were also related to the higher order frequency response func- 
tions, and can be connected through the cumulant expressions. The general relations of 
these functions were summarized. The achievements of J. F. Dalzell on the application of 
polyspectra and in the derivation of the probability distributions of extremes of seakeep- 
ing data were introduced at length. At the end of Part III, as an extension of the model 
fitting approach treated in Part II, a few trials on nonlinear parametric models proposed 
by several statisticians were introduced. Some of these trials look promising and are still 
under active development. However, as has been concluded in Chapter 14, we need more 
experience with such applications. 
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